{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,13]],"date-time":"2026-08-13T21:55:12Z","timestamp":1786658112808,"version":"build-2736575974"},"reference-count":65,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2023,3,23]],"date-time":"2023-03-23T00:00:00Z","timestamp":1679529600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Doctorate Scholarship","award":["CONICYT 2019"],"award-info":[{"award-number":["CONICYT 2019"]}]},{"name":"National Doctorate Scholarship","award":["1220556"],"award-info":[{"award-number":["1220556"]}]},{"name":"ANID\/ATE220023 Project","award":["CONICYT 2019"],"award-info":[{"award-number":["CONICYT 2019"]}]},{"name":"ANID\/ATE220023 Project","award":["1220556"],"award-info":[{"award-number":["1220556"]}]},{"name":"FONDECYT Regular Research Project","award":["CONICYT 2019"],"award-info":[{"award-number":["CONICYT 2019"]}]},{"name":"FONDECYT Regular Research Project","award":["1220556"],"award-info":[{"award-number":["1220556"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Software Defined Networking (SDN) is a communication alternative to increase the scalability and resilience of microgrid hierarchical control. The common architecture has a centralized and monolithic topology, where the controller is highly susceptible to latency problems, resiliency, and scalability issues. This paper proposes a novel and intelligent control network to improve the performance of microgrid communications, solving the typical drawback of monolithic SDN controllers. The SDN controller\u2019s functionalities are segregated into microservices groups and distributed through a bare-metal Kubernetes cluster. Results are presented from PLECS hardware in the loop simulation to validate the seamless transition between standard hierarchical control to the SDN networked microgrid. The microservices significantly impact the performance of the SDN controller, decreasing the latency by 10.76% compared with a monolithic architecture. Furthermore, the proposed approach demonstrates a 42.23% decrease in packet loss versus monolithic topologies and a 53.41% reduction in recovery time during failures. Combining Kubernetes with SDN microservices can eliminate the single point of failure in hierarchical control, improve application recovery time, and enhance containerization benefits, including security and portability. This proposal represents a reference framework for future edge computing and intelligent control approaches in networked microgrids.<\/jats:p>","DOI":"10.3390\/s23073395","type":"journal-article","created":{"date-parts":[[2023,3,24]],"date-time":"2023-03-24T03:16:46Z","timestamp":1679627806000},"page":"3395","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Moving Microgrid Hierarchical Control to an SDN-Based Kubernetes Cluster: A Framework for Reliable and Flexible Energy Distribution"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9149-5074","authenticated-orcid":false,"given":"Ricardo","family":"P\u00e9rez","sequence":"first","affiliation":[{"name":"Department of Computer Science, Faculty of Engineering, Universidad de Talca, Curic\u00f3 3341717, Chile"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4353-2088","authenticated-orcid":false,"given":"Marco","family":"Rivera","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Faculty of Engineering, Universidad de Talca, Curic\u00f3 3341717, Chile"},{"name":"Department of Electrical and Electronic Engineering, Faculty of Engineering, University of Nottingham, Nottingham, NG7 2GT, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1946-0053","authenticated-orcid":false,"given":"Yamisleydi","family":"Salgueiro","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering, Faculty of Engineering, Universidad de Talca, Curic\u00f3 3341717, Chile"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1752-1625","authenticated-orcid":false,"given":"Carlos R.","family":"Baier","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Faculty of Engineering, Universidad de Talca, Curic\u00f3 3341717, Chile"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0307-581X","authenticated-orcid":false,"given":"Patrick","family":"Wheeler","sequence":"additional","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Faculty of Engineering, University of Nottingham, Nottingham, NG7 2GT, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"106629","DOI":"10.1016\/j.ijepes.2020.106629","article-title":"Comprehensive power flow modelling of hierarchically controlled AC\/DC hybrid islanded microgrids","volume":"127","author":"Tinajero","year":"2021","journal-title":"Int. J. Electr. Power Energy Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2427","DOI":"10.1109\/TPEL.2016.2569597","article-title":"Review of active and reactive power sharing strategies in hierarchical controlled microgrids","volume":"32","author":"Han","year":"2016","journal-title":"IEEE Trans. Power Electron."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"107531","DOI":"10.1016\/j.epsr.2021.107531","article-title":"Improved droop control strategy for parallel connected power electronic converter based distributed generation sources in an Islanded Microgrid","volume":"201","author":"Kulkarni","year":"2021","journal-title":"Electr. Power Syst. Res."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"P\u00e9rez-Guzm\u00e1n, R.E., Salgueiro-Sicilia, Y., and Rivera, M. (2017, January 18\u201320). Communications in smart grids. Proceedings of the 2017 CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON), Pucon, Chile.","DOI":"10.1109\/CHILECON.2017.8229561"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"7025","DOI":"10.1109\/TIE.2015.2436879","article-title":"Secondary frequency and voltage control of islanded microgrids via distributed averaging","volume":"62","author":"Shafiee","year":"2015","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"6482","DOI":"10.1109\/TPEL.2019.2951694","article-title":"On the secondary control architectures of AC microgrids: An overview","volume":"35","author":"Khayat","year":"2019","journal-title":"IEEE Trans. Power Electron."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ferreira, D., Silva, S., Silva, W., Brandao, D., Bergna, G., and Tedeschi, E. (2022). Overview of Consensus Protocol and Its Application to Microgrid Control. Energies, 15.","DOI":"10.3390\/en15228536"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1016\/j.isatra.2022.02.039","article-title":"A switching event-triggered resilient control scheme for primary and secondary levels in AC microgrids","volume":"127","author":"Shan","year":"2022","journal-title":"ISA Trans."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"922","DOI":"10.1109\/TEC.2014.2362191","article-title":"Hierarchical control for multiple DC-microgrids clusters","volume":"29","author":"Shafiee","year":"2014","journal-title":"IEEE Trans. Energy Convers."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2586","DOI":"10.1109\/COMST.2020.3023963","article-title":"Distributed control and communication strategies in networked microgrids","volume":"22","author":"Zhou","year":"2020","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"102947","DOI":"10.1016\/j.jnca.2020.102947","article-title":"A survey on network forwarding in Software-Defined Networking","volume":"176","author":"Yang","year":"2021","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"98973","DOI":"10.1109\/ACCESS.2021.3095317","article-title":"Software-defined power grids: A survey on opportunities and taxonomy for microgrids","volume":"9","author":"Ndiaye","year":"2021","journal-title":"IEEE Access"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1251","DOI":"10.1016\/j.apenergy.2017.06.006","article-title":"Enabling resilient distributed power sharing in networked microgrids through software defined networking","volume":"210","author":"Ren","year":"2018","journal-title":"Appl. Energy"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2826","DOI":"10.1109\/TSG.2016.2589903","article-title":"Enabling Resilient Microgrid Through Programmable Network","volume":"8","author":"Ren","year":"2017","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"5258","DOI":"10.1109\/TSG.2018.2879727","article-title":"Software-Defined Microgrid Control for Resilience Against Denial-of-Service Attacks","volume":"10","author":"Danzi","year":"2019","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/MCOM.001.1900063","article-title":"Toward disaggregating the SDN control plane","volume":"57","author":"Comer","year":"2019","journal-title":"IEEE Commun. Mag."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10922-021-09631-7","article-title":"MSN: A Playground Framework for Design and Evaluation of MicroServices-Based sdN Controller","volume":"30","author":"Arzo","year":"2022","journal-title":"J. Netw. Syst. Manag."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"70850","DOI":"10.1109\/ACCESS.2022.3188311","article-title":"Towards Software-Defined Networking-based IoT Frameworks: A Systematic Literature Review, Taxonomy, Open Challenges and Prospects","volume":"10","author":"Siddiqui","year":"2022","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"170070","DOI":"10.1109\/ACCESS.2020.3023974","article-title":"Comprehensive review of SDN controller placement strategies","volume":"8","author":"Isong","year":"2020","journal-title":"IEEE Access"},{"key":"ref_20","unstructured":"Nippon Telegraph and Telephone Corporation (NTT) (2022, December 22). Ryu SDN Controller. Available online: https:\/\/ryu-sdn.org\/."},{"key":"ref_21","unstructured":"(2022, June 03). OpenDaylight (ODL) Controller. Available online: https:\/\/www.opendaylight.org\/."},{"key":"ref_22","unstructured":"ONOS Project Community (2023, January 18). Open Network Operating System (ONOS). Available online: https:\/\/opennetworking.org\/onos\/."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Markelov, A. (2022). Certified OpenStack Administrator Study Guide, Springer.","DOI":"10.1007\/978-1-4842-8804-7"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"H\u00f6lscher, A., Asplund, M., and Boeira, F. (July, January 27). Evaluation of an SDN-based Microservice Architecture. Proceedings of the 2022 IEEE 8th International Conference on Network Softwarization (NetSoft), Milan, Italy.","DOI":"10.1109\/NetSoft54395.2022.9844094"},{"key":"ref_25","unstructured":"Open Network Foundation (2022, October 22). Open Network Operating System (ONOS). Available online: https:\/\/docs.onosproject.org\/."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/j.comcom.2021.01.018","article-title":"SDN\/NFV architectures for edge-cloud oriented IoT: A systematic review","volume":"169","author":"Ray","year":"2021","journal-title":"Comput. Commun."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"229117","DOI":"10.1109\/ACCESS.2020.3045563","article-title":"SDN enhanced resource orchestration of containerized edge applications for industrial IoT","volume":"8","author":"Okwuibe","year":"2020","journal-title":"IEEE Access"},{"key":"ref_28","first-page":"5037702","article-title":"Software-Defined Networks for Optical Networks Using Flexible Orchestration: Advances, Challenges, and Opportunities","volume":"2022","author":"Tchao","year":"2022","journal-title":"J. Comput. Netw. Commun."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3610","DOI":"10.1016\/j.rser.2017.10.101","article-title":"Current challenges and future trends in the field of communication architectures for microgrids","volume":"82","author":"Marzal","year":"2018","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Abbasi, M., Abbasi, E., Li, L., Aguilera, R.P., Lu, D., and Wang, F. (2023). Review on the Microgrid Concept, Structures, Components, Communication Systems, and Control Methods. Energies, 16.","DOI":"10.3390\/en16010484"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"L\u00e9vy, L.N., Bosom, J., Guerard, G., Amor, S.B., Bui, M., and Tran, H. (2022). DevOps Model Appproach for Monitoring Smart Energy Systems. Energies, 15.","DOI":"10.3390\/en15155516"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Johansson, B., R\u00e5gberger, M., Nolte, T., and Papadopoulos, A.V. (2022, January 22\u201325). Kubernetes orchestration of high availability distributed control systems. Proceedings of the 2022 IEEE International Conference on Industrial Technology (ICIT), Shanghai, China.","DOI":"10.1109\/ICIT48603.2022.10002757"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhu, C., Han, B., and Zhao, Y. (2020, January 4\u20136). A Comparative Study of Spark on the bare metal and Kubernetes. Proceedings of the 2020 6th International Conference on Big Data and Information Analytics (BigDIA), Shenzhen, China.","DOI":"10.1109\/BigDIA51454.2020.00027"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10723-021-09545-3","article-title":"Opportunistic deployment of distributed edge clouds for latency-critical applications","volume":"19","author":"Huedo","year":"2021","journal-title":"J. Grid Comput."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1109\/LNET.2023.3235106","article-title":"Benefits of Pod dimensioning with best-effort resources in bare metal cloud native deployments","volume":"5","author":"Tonini","year":"2023","journal-title":"IEEE Netw. Lett."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Klos, A., Rosenbaum, M., and Schiffmann, W. (2021, January 17\u201321). Scalable and highly available multi-objective neural architecture search in bare metal kubernetes cluster. Proceedings of the 2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Portland, OR, USA.","DOI":"10.1109\/IPDPSW52791.2021.00094"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"111389","DOI":"10.1109\/ACCESS.2022.3215434","article-title":"Microgrid Power Sharing Framework for Software Defined Networking and Cybersecurity Analysis","volume":"10","author":"Rivera","year":"2022","journal-title":"IEEE Access"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Yadav, G., Joshi, D., Gopinath, L., and Soni, M.K. (2022). Reliability and Availability Optimization of Smart Microgrid Using Specific Configuration of Renewable Resources and Considering Subcomponent Faults. Energies, 15.","DOI":"10.3390\/en15165994"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/s10922-020-09575-4","article-title":"Scalability, Consistency, Reliability and Security in SDN Controllers: A Survey of Diverse SDN Controllers","volume":"29","author":"Ahmad","year":"2021","journal-title":"J. Netw. Syst. Manag."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"108369","DOI":"10.1016\/j.comnet.2021.108369","article-title":"Multiple-level threshold load balancing in distributed SDN controllers","volume":"198","author":"Mokhtar","year":"2021","journal-title":"Comput. Netw."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Gupta, N., Maashi, M.S., Tanwar, S., Badotra, S., Aljebreen, M., and Bharany, S. (2022). A Comparative Study of Software Defined Networking Controllers Using Mininet. Electronics, 11.","DOI":"10.3390\/electronics11172715"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1109\/TIE.2010.2066534","article-title":"Hierarchical control of droop-controlled AC and DC microgrids\u2014A general approach toward standardization","volume":"58","author":"Guerrero","year":"2010","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_43","unstructured":"Garces, L. (2022, December 03). Microgrid in Island Operation. Available online: https:\/\/www.plexim.com\/support\/application-examples\/1259."},{"key":"ref_44","unstructured":"Garces, L.J., Liu, Y., and Bose, S. (2007). System and Method for Integrating Wind and Hydroelectric Generation and Pumped Hydro Energy Storage Systems. (7,239,035), U.S. Patent."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"100491","DOI":"10.1016\/j.segan.2021.100491","article-title":"Energy exchange cooperative model in SDN-based interconnected multi-microgrids","volume":"27","author":"Zargar","year":"2021","journal-title":"Sustain. Energy Grids Netw."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"107981","DOI":"10.1016\/j.comnet.2021.107981","article-title":"Hybrid SDN evolution: A comprehensive survey of the state-of-the-art","volume":"192","author":"Khorsandroo","year":"2021","journal-title":"Comput. Netw."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1467","DOI":"10.1109\/TNSE.2021.3060372","article-title":"Traffic Engineering to Minimize the Number of Rules in SDN Datacenters","volume":"8","author":"Biswas","year":"2021","journal-title":"IEEE Trans. Netw. Sci. Eng."},{"key":"ref_48","first-page":"770","article-title":"A Research Review of OpenFlow for Datacenter Networking","volume":"11","year":"2022","journal-title":"IEEE Access"},{"key":"ref_49","unstructured":"(2022, November 25). Available online: https:\/\/k3s.io\/."},{"key":"ref_50","unstructured":"Labs, R. (2022, November 25). Rancher: Enterprise Kubernetes Management. Available online: https:\/\/www.rancher.com\/."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Wazirali, R., Ahmad, R., and Alhiyari, S. (2021). SDN-OpenFlow Topology Discovery: An Overview of Performance Issues. Appl. Sci.-Basel, 11.","DOI":"10.3390\/app11156999"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"286","DOI":"10.35833\/MPCE.2021.000669","article-title":"Architecture, Control, and Implementation of Networked Microgrids for Future Distribution Systems","volume":"10","author":"Yan","year":"2022","journal-title":"J. Mod. Power Syst. Clean Energy"},{"key":"ref_53","unstructured":"Siva Ananmalay, J.A., and Barton, D. (2022, July 16). Open Networking Foundation. Available online: https:\/\/opennetworking.org\/."},{"key":"ref_54","unstructured":"Vachuska, T. (2022, December 03). ONOS Helm Charts. Available online: https:\/\/github.com\/onosproject\/onos-helm-charts."},{"key":"ref_55","unstructured":"Vachuska, T., and Halterman, J. (2022, July 16). Atomix-Controller: Kubernetes Controller for Atomix 4. Available online: https:\/\/github.com\/atomix\/atomix-controller."},{"key":"ref_56","unstructured":"Open Network Foundation (2022, December 07). Deploying Onos-Config. Available online: https:\/\/docs.onosproject.org\/onos-config\/docs\/deployment\/."},{"key":"ref_57","unstructured":"P\u00e9rez, R. (2022, December 03). Deploy HA Kubernetes Cluster for SDN Microgrid Hierarchical Control. Available online: https:\/\/github.com\/ricardopg1987\/kubernetes-rpi."},{"key":"ref_58","unstructured":"(2022, September 25). Available online: https:\/\/docs.k3s.io\/installation\/ha-embedded."},{"key":"ref_59","unstructured":"KubeSphere (2022, September 25). Set up an HA Kubernetes Cluster Using Keepalived and HAproxy. Available online: https:\/\/kubesphere.io\/docs\/v3.3\/installing-on-linux\/high-availability-configurations\/set-up-ha-cluster-using-keepalived-haproxy\/."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Zhang, Z. (2022, January 19\u201321). A comparison of low-speed communication modes. Proceedings of the International Conference on Network Communication and Information Security (ICNCIS 2021), Qingdao, China.","DOI":"10.1117\/12.2628425"},{"key":"ref_61","unstructured":"(2023, March 17). ZodiacFX Communication Device. Available online: https:\/\/www.cryptomuseum.com\/radio\/zodiac\/."},{"key":"ref_62","unstructured":"Muhammad, A., Saqib, M., and Song, W.C. (2021, January 18\u201320). Sensor Virtualization and Data Orchestration in Internet of Vehicles (IoV). Proceedings of the 2021 IFIP\/IEEE International Symposium on Integrated Network Management (IM), Bordeaux, France."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"11320","DOI":"10.1109\/TPEL.2019.2896955","article-title":"High-bandwidth secondary voltage and frequency control of vsc-based ac microgrid","volume":"34","author":"Heydari","year":"2019","journal-title":"IEEE Trans. Power Electron."},{"key":"ref_64","first-page":"6304","article-title":"Model predictive control of power converters for robust and fast operation of AC microgrids","volume":"33","year":"2017","journal-title":"IEEE Trans. Power Electron."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Villal\u00f3n, A., Rivera, M., Salgueiro, Y., Mu noz, J., Dragi\u010devi\u0107, T., and Blaabjerg, F. (2020). Predictive control for microgrid applications: A review study. Energies, 13.","DOI":"10.3390\/en13102454"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3395\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:01:36Z","timestamp":1760122896000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3395"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,23]]},"references-count":65,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["s23073395"],"URL":"https:\/\/doi.org\/10.3390\/s23073395","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,23]]}}}