{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T19:29:18Z","timestamp":1775849358407,"version":"3.50.1"},"reference-count":51,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T00:00:00Z","timestamp":1761868800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>Big Data Analytics is vital for power grids, as it empowers informed decision-making, anticipates potential operational and maintenance issues, optimizes grid management, supports renewable energy integration, ultimately reduces costs, improves customer service, monitors consumer behavior, and offers new services. This paper describes the AI\u2013Big Data Analytics Architecture based on a data lake architecture that uses a reduced and customized set of Hadoop and Spark as a cost-effective, on-premises alternative for advanced data analytics in power systems. As a case study, a comparative analysis of electricity price forecasting models in the day-ahead market for nodes of the Mexican national electrical system using statistical, machine learning, and deep learning models, is presented. To build and select the best forecasting model, a data science and machine learning methodology is used. The results show that the Gradient Boosting and Support Vector Regression models presented the best performance, with a Mean Absolute Percentage Error (MAPE) between 1% and 4% for five-day-ahead electricity price forecasting. The implementation of the best forecasting model into the Big Data Analytics Platform allows the automation of the calculation of the local electricity price forecast per node (every 24, 72, or 120 h) and its display in a comparative dashboard with actual and forecasted data for decision-making on demand. The proposed architecture is a valuable tool that allows the future implementation of intelligent energy forecasting models in power grids, such as load demand, fuel prices, power generation, and consumption, among others.<\/jats:p>","DOI":"10.3390\/bdcc9110272","type":"journal-article","created":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T05:28:43Z","timestamp":1761888523000},"page":"272","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["AI\u2013Big Data Analytics Platform for Energy Forecasting in Modern Power Systems"],"prefix":"10.3390","volume":"9","author":[{"given":"Martin","family":"Santos-Dominguez","sequence":"first","affiliation":[{"name":"Instituto Nacional de Electricidad y Energias Limpias, Cuernavaca 62490, Mexico"}]},{"given":"Nicasio","family":"Hernandez Flores","sequence":"additional","affiliation":[{"name":"Instituto Nacional de Electricidad y Energias Limpias, Cuernavaca 62490, Mexico"}]},{"given":"Isaac Alberto","family":"Parra-Ramirez","sequence":"additional","affiliation":[{"name":"Instituto Nacional de Electricidad y Energias Limpias, Cuernavaca 62490, Mexico"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0764-045X","authenticated-orcid":false,"given":"Gustavo","family":"Arroyo-Figueroa","sequence":"additional","affiliation":[{"name":"Instituto Nacional de Electricidad y Energias Limpias, Cuernavaca 62490, Mexico"}]}],"member":"1968","published-online":{"date-parts":[[2025,10,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Liao, H., Michalenko, E., and Vegunta, S.C. (2023). Review of Big Data Analytics for Smart Electrical Energy Systems. Energies, 16.","DOI":"10.3390\/en16083581"},{"key":"ref_2","first-page":"1","article-title":"Editorial: An overview of Applied Artificial Intelligence in Power Grids","volume":"15","year":"2024","journal-title":"Int. J. Comb. Optim. Probl. Inform."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3125","DOI":"10.1109\/TSG.2018.2818167","article-title":"Review of Smart Meter Data Analytics: Applications, Methodologies, and Challenges","volume":"10","author":"Wang","year":"2019","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.rser.2015.11.050","article-title":"Big data driven smart energy management: From big data to big insights","volume":"56","author":"Zhou","year":"2016","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3844","DOI":"10.1109\/ACCESS.2016.2580581","article-title":"Energy big data: A survey","volume":"4","author":"Jiang","year":"2016","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Guerrero-Prado, J.S., Alfonso-Morales, W., Caicedo-Bravo, E., Zayas-P\u00e9rez, B., and Espinosa-Reza, A. (2020). The Power of Big Data and Data Analytics for AMI Data: A Case Study. Sensors, 20.","DOI":"10.3390\/s20113289"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1186\/s42162-018-0007-5","article-title":"Big data analytics in smart grids: A review","volume":"1","author":"Zhang","year":"2018","journal-title":"Energy Inform."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"106788","DOI":"10.1016\/j.epsr.2020.106788","article-title":"Big data analytics for future electricity grids","volume":"189","author":"Kezunovic","year":"2020","journal-title":"Electr. Power Syst. Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"59564","DOI":"10.1109\/ACCESS.2020.3041178","article-title":"Smart Grid Big Data Analytics: Survey of Technologies, Techniques, and Applications","volume":"9","author":"Syed","year":"2021","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Escobedo, G., Jacome, N., and Arroyo-Figueroa, G. (2017, January 24\u201326). Big Data & Analytics to Support the Renewable Energy Integration of Smart Grids\u2014Case Study: Power Solar Generation. Proceedings of the 2nd International Conference on Internet of Things, Big Data and Security IoTBDS, Porto, Portugal.","DOI":"10.5220\/0006297502670275"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Alhamrouni, I., Abdul Kahar, N.H., Salem, M., Swadi, M., Zahroui, Y., Kadhim, D.J., Mohamed, F.A., and Alhuyi Nazari, M. (2024). A Comprehensive Review on the Role of Artificial Intelligence in Power System Stability, Control, and Protection: Insights and Future Directions. Appl. Sci., 14.","DOI":"10.3390\/app14146214"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1186\/s40537-020-00329-2","article-title":"Predictive big data analytics for supply chain demand forecasting: Methods, applications, and research opportunities","volume":"7","author":"Seyedan","year":"2020","journal-title":"J. Big Data"},{"key":"ref_13","first-page":"104011","article-title":"Smart grid and application of big data: Opportunities and challenges","volume":"71","author":"Mohanty","year":"2024","journal-title":"Sustain. Energy Technol. Assess."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"111459","DOI":"10.1016\/j.rser.2021.111459","article-title":"Artificial intelligence techniques for enabling Big Data services in distribution networks: A review","volume":"150","year":"2021","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Huang, L. (2023, January 14\u201316). Intelligent Condition Monitoring and Fault Diagnosis of Generator based on Internet of Things and Big Data Technology. Proceedings of the 2023 IEEE 13th International Conference on Electronics Information and Emergency Communication (ICEIEC), Beijing, China.","DOI":"10.1109\/ICEIEC58029.2023.10200780"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wang, X., and Duan, Z. (2023, January 25\u201327). Application of Artificial Intelligence Technology in Power Equipment Condition Prediction and Maintenance. Proceedings of the International Conference on Power, Electrical Engineering, Electronics and Control (PEEEC), Athens, Greece.","DOI":"10.1109\/PEEEC60561.2023.00023"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1016\/j.ijepes.2014.12.036","article-title":"Forecasting electricity consumption: A comparison of regression analysis, neural networks and least squares support vector machines","volume":"67","author":"Kaytez","year":"2015","journal-title":"Int. J. Electr. Power Energy Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"112128","DOI":"10.1016\/j.rser.2022.112128","article-title":"Data-driven probabilistic machine learning in sustainable smart energy\/smart energy systems: Key developments, challenges, and future research opportunities in the context of smart grid paradigm","volume":"160","author":"Ahmad","year":"2022","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1016\/j.bdr.2015.03.003","article-title":"Big Data Analytics for Dynamic Energy Management in Smart Grids","volume":"2","author":"Diamantoulakis","year":"2015","journal-title":"Big Data Res."},{"key":"ref_20","first-page":"376","article-title":"Energy Forecasting: A Review and Outlook","volume":"7","author":"Hong","year":"2020","journal-title":"J. Power Energy"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1016\/j.scs.2017.08.009","article-title":"Electrical load forecasting models: A critical systematic review","volume":"35","author":"Kuster","year":"2017","journal-title":"Sustain. Cities Soc."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.rser.2015.04.081","article-title":"Ensemble methods for wind and solar power forecasting\u2014A state-of-the-art review","volume":"50","author":"Ren","year":"2015","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Klyuev, R.V., Morgoev, I.D., Morgoeva, A.D., Gavrina, O.A., Martyushev, N.V., Efremenkov, E.A., and Mengxu, Q. (2022). Methods of Forecasting Electric Energy Consumption: A Literature Review. Energies, 15.","DOI":"10.3390\/en15238919"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"116983","DOI":"10.1016\/j.apenergy.2021.116983","article-title":"Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark","volume":"293","author":"Lago","year":"2021","journal-title":"Appl. Energy"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1016\/j.apenergy.2018.02.069","article-title":"Forecasting spot electricity prices: Deep learning approaches and empirical comparison of traditional algorithms","volume":"221","author":"Lago","year":"2018","journal-title":"Appl. Energy"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1016\/j.apenergy.2016.12.130","article-title":"Electricity price forecasting by a hybrid model, combining wavelet transform, ARMA and kernel-based extreme learning machine methods","volume":"190","author":"Yang","year":"2017","journal-title":"Appl. Energy"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"K\u0131l\u0131\u00e7, D.K., Nielsen, P., and Thibbotuwawa, A. (2024). Intraday Electricity Price Forecasting via LSTM and Trading Strategy for the Power Market: A Case Study of the West Denmark DK1 Grid Region. Energies, 17.","DOI":"10.3390\/en17122909"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Dudek, G. (2022). A Comprehensive Study of Random Forest for Short-Term Load Forecasting. Energies, 15.","DOI":"10.3390\/en15207547"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zhao, X., Li, Q., Xue, W., Zhao, Y., Zhao, H., and Guo, S. (2022). Research on Ultra-Short-Term Load Forecasting Based on Real-Time Electricity Price and Window-Based XGBoost Model. Energies, 15.","DOI":"10.3390\/en15197367"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Narajewski, M. (2022). Probabilistic forecasting of German electricity imbalance prices. Energies, 15.","DOI":"10.3390\/en15144976"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"101436","DOI":"10.1016\/j.esr.2024.101436","article-title":"Electricity Price Forecasting in the Irish Balancing Market","volume":"54","author":"Collins","year":"2024","journal-title":"Energy Strategy Rev."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zahid, M., Ahmed, F., Javaid, N., Abbasi, R.A., Zainab Kazmi, H.S., Javaid, A., Bilal, M., Akbar, M., and Ilahi, M. (2019). Electricity price and load forecasting using enhanced convolutional neural network and enhanced support vector regression in smart grids. Electronics, 8.","DOI":"10.3390\/electronics8020122"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"126011","DOI":"10.1016\/j.energy.2022.126011","article-title":"Daily electricity price forecasting using artificial intelligence models in the Iranian electricity market","volume":"263","author":"Heidarpanah","year":"2023","journal-title":"Energy"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Sarnovsky, M., Bednar, P., and Smatana, M. (2018). Big Data Processing and Analytics Platform Architecture for Process Industry Factories. Big Data Cogn. Comput., 2.","DOI":"10.3390\/bdcc2010003"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1016\/j.ins.2014.01.015","article-title":"Data-intensive applications, challenges, techniques and technologies: A survey on Big Data","volume":"275","author":"Chen","year":"2014","journal-title":"Inf. Sci."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"5467","DOI":"10.1016\/j.egyr.2021.08.133","article-title":"A comprehensive review: Machine learning and its application in integrated power system","volume":"7","author":"Pravin","year":"2021","journal-title":"Energy Rep."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"101582","DOI":"10.1016\/j.esr.2024.101582","article-title":"Survey of technologies, techniques, and applications for big data analytics in smart energy hub","volume":"56","author":"Sedhom","year":"2024","journal-title":"Energy Strategy Rev."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Ajah, I.A., and Nweke, H.F. (2019). Big Data and Business Analytics: Trends, Platforms, Success Factors and Applications. Big Data Cogn. Comput., 3.","DOI":"10.3390\/bdcc3020032"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Nambiar, A., and Mundra, D. (2022). An Overview of Data Warehouse and Data Lake in Modern Enterprise Data Management. Big Data Cogn. Comput., 6.","DOI":"10.3390\/bdcc6040132"},{"key":"ref_40","unstructured":"Buyya, R., Calheiros, R.N., and Dastjerdi, A.V. (2023). Big Data Principles and Paradigmes, Morgan Kaufmann."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"113","DOI":"10.13053\/rcs-122-1-9","article-title":"Design of a Technology Management Infrastructure for Large Volumes of Data in an Intelligent Power Network","volume":"122","author":"Escobedo","year":"2016","journal-title":"Res. Comput. Sci."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Amaya-Sanchez, Q., Argumedo, M.J.D.M., Aguilar-Lasserre, A.A., Reyes Martinez, O.A., and Arroyo-Figueroa, G. (2024). Fault Diagnosis in Power Generators: A Comparative Analysis of Machine Learning Models. Big Data Cogn. Comput, 8.","DOI":"10.3390\/bdcc8110145"},{"key":"ref_43","unstructured":"(2024, May 23). Programa Sectorial de Energ\u00eda 2020\u20132024. Secretaria de Energia (SENER). Available online: https:\/\/www.gob.mx\/cms\/uploads\/attachment\/file\/562631\/PS_SENER_CACEC-DOF_08-07-2020.pdf."},{"key":"ref_44","unstructured":"(2024, May 23). Sistema de Informaci\u00f3n del Mercado (SIM). Centro Nacional de Control de Energ\u00eda (CENACE). Available online: https:\/\/www.gob.mx\/cenace\/acciones-y-programas\/sistema-de-informacion-de-mercado-sim."},{"key":"ref_45","unstructured":"Nielsen, A. (2019). Practical Time Series Analysis: Prediction with Statistics and Machine Learning. Oreilly Media Inc."},{"key":"ref_46","unstructured":"(2024, July 27). Anaconda Software Distribution. Conda (Version 3-13.5). Available online: https:\/\/anaconda.org\/anaconda\/python."},{"key":"ref_47","unstructured":"Oliphant, T.E. (2024, July 27). A Guide to NumPy. Volume 1. Available online: https:\/\/web.mit.edu\/dvp\/Public\/numpybook.pdf."},{"key":"ref_48","first-page":"1","article-title":"Pandas: A foundational python library for data analysis and statistics","volume":"14","author":"McKinney","year":"2011","journal-title":"Python High Perform. Sci. Comput."},{"key":"ref_49","first-page":"2825","article-title":"Scikit-learn: Machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Mystakidis, A., Koukaras, P., Tsalikidis, N., Ioannidis, D., and Tjortjis, C. (2024). Energy Forecasting: A Comprehensive Review of Techniques and Technologies. Energies, 17.","DOI":"10.3390\/en17071662"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"882","DOI":"10.3390\/forecast4040048","article-title":"Precision and Reliability of Forecasts Performance Metrics","volume":"4","author":"Agard","year":"2022","journal-title":"Forecasting"}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/11\/272\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T05:59:04Z","timestamp":1761890344000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/11\/272"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,31]]},"references-count":51,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["bdcc9110272"],"URL":"https:\/\/doi.org\/10.3390\/bdcc9110272","relation":{},"ISSN":["2504-2289"],"issn-type":[{"value":"2504-2289","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,31]]}}}