{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T15:52:11Z","timestamp":1774021931490,"version":"3.50.1"},"reference-count":44,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,10,21]],"date-time":"2021-10-21T00:00:00Z","timestamp":1634774400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,10,21]],"date-time":"2021-10-21T00:00:00Z","timestamp":1634774400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Alma Mater Studiorum - Universit\u00e0 di Bologna"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Netw Syst Manage"],"published-print":{"date-parts":[[2022,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Software-defined networking decouples control and data plane in <jats:italic>softwarized<\/jats:italic> networks. This allows for centralized management of the network, but complete centralization of the controller functions raises potential issues related to failure, latency, and scalability. Distributed controller deployment is adopted to optimize scalability and latency problems. However, existing controllers are monolithic, resulting in code inefficiency for distributed deployment. Some seminal ongoing efforts have been proposed with the idea of disaggregating the SDN controller architecture into an assembly of various subsystems, each of which can be responsible for a certain controller task. These subsystems are typically implemented as microservices and deployed as virtual network functions, in particular as Docker Containers. This enables flexible deployment of controller functions. However, these proposals (e.g., <jats:inline-formula><jats:alternatives><jats:tex-math>$$\\mu$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mi>\u03bc<\/mml:mi>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula>ONOS) are still in their early stage of design and development, so that a full decomposition of the SDN controller is not been available yet. To fill that gap, this article derives some important design guidelines to decompose an SDN controller into a set of microservices. Next, it also proposes a microservices-based decomposed controller architecture, foreseeing communications issues between the controller sub-functions. These design and performance considerations are also proven via the implementation of the proposed architecture as a solution, called Micro-Services based SDN controller (MSN), based on the Ryu SDN controller. Moreover, MSN includes different network communication protocols, such as gRPC, WebSocket, and REST-API. Finally, we show experimental results that highlight the robustness and latency of the system on a networking testbed. Collected results prove the main pros and cons of each network communication protocol and an evaluation of our proposal in terms of system resilience, scalability and latency.<\/jats:p>","DOI":"10.1007\/s10922-021-09631-7","type":"journal-article","created":{"date-parts":[[2021,10,21]],"date-time":"2021-10-21T13:03:48Z","timestamp":1634821428000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["MSN: A Playground Framework for Design and Evaluation of MicroServices-Based sdN Controller"],"prefix":"10.1007","volume":"30","author":[{"given":"Sisay Tadesse","family":"Arzo","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3824-197X","authenticated-orcid":false,"given":"Domenico","family":"Scotece","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Riccardo","family":"Bassoli","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel","family":"Barattini","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fabrizio","family":"Granelli","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luca","family":"Foschini","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Frank H. P.","family":"Fitzek","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,10,21]]},"reference":[{"key":"9631_CR1","unstructured":"Schenker, S.: The future of networking, the past of protocols, Mar. 2021. https:\/\/youtu.be\/YHeyuD89n1Y"},{"issue":"2","key":"9631_CR2","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1109\/MCOM.2013.6461195","volume":"51","author":"H Kim","year":"2013","unstructured":"Kim, H., Feamster, N.: Improving network management with software defined networking. IEEE Commun. Mag. 51(2), 114\u2013119 (2013)","journal-title":"IEEE Commun. Mag."},{"key":"9631_CR3","unstructured":"Open Networking Foundation: Software-Defined Networking: The New Norm for Networks (2012)"},{"issue":"2","key":"9631_CR4","first-page":"69","volume":"38","author":"N McKeown","year":"2008","unstructured":"McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G., Peterson, L., Rexford, J., Shenker, S., Turner, J.: Openflow: Enabling innovation in campus networks in SIGCOMM Comput. Commun. Rev. 38(2), 69\u201374 (2008)","journal-title":"Commun. Rev."},{"issue":"1","key":"9631_CR5","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1109\/COMST.2017.2782482","volume":"20","author":"F Bannour","year":"2018","unstructured":"Bannour, F., Souihi, S., Mellouk, A.: Distributed SDN control: survey, taxonomy, and challenges. IEEE Commun. Surv. Tutor. 20(1), 333\u2013354 (2018)","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"9631_CR6","unstructured":"Ryu: Ryu SDN Framework. 2021. https:\/\/ryu-sdn.org\/"},{"key":"9631_CR7","unstructured":"ONOS: Open Network Operating System. 2021. https:\/\/opennetworking.org\/onos\/"},{"key":"9631_CR8","unstructured":"OpenDaylight: OpenDaylight and Open Networking ecosystem. 2021. https:\/\/www.opendaylight.org\/"},{"key":"9631_CR9","unstructured":"Floodlight: Floodlight Controller. 2021. https:\/\/floodlight.atlassian.net\/wiki\/spaces\/floodlightcontroller\/overview"},{"key":"9631_CR10","doi-asserted-by":"publisher","first-page":"106984","DOI":"10.1016\/j.comnet.2019.106984","volume":"167","author":"AA Barakabitze","year":"2020","unstructured":"Barakabitze, A.A., Ahmad, A., Mijumbi, R., Hines, A.: 5G network slicing using SDN and NFV: a survey of taxonomy, architectures and future challenges. Comput. Netw. 167, 106984 (2020). https:\/\/doi.org\/10.1016\/j.comnet.2019.106984","journal-title":"Comput. Netw."},{"issue":"5","key":"9631_CR11","doi-asserted-by":"publisher","first-page":"138","DOI":"10.1109\/MCOM.2016.7470948","volume":"54","author":"M Maier","year":"2016","unstructured":"Maier, M., Chowdhury, M., Rimal, B.P., Van, D.P.: The tactile internet: vision, recent progress, and open challenges. IEEE Commun. Mag. 54(5), 138\u2013145 (2016). https:\/\/doi.org\/10.1109\/MCOM.2016.7470948","journal-title":"IEEE Commun. Mag."},{"key":"9631_CR12","unstructured":"5G-PPP: White paper: 5g and the factories of the future. 2021. https:\/\/5g-ppp.eu\/wp-content\/uploads\/2014\/02\/5G-PPP-White-Paper-on-Factories-of-the-Future-Vertical-Sector.pdf"},{"issue":"3","key":"9631_CR13","doi-asserted-by":"publisher","first-page":"2429","DOI":"10.1109\/COMST.2018.2815638","volume":"20","author":"I Afolabi","year":"2018","unstructured":"Afolabi, I., Taleb, T., Samdanis, K., Ksentini, A., Flinck, H.: Network slicing and softwarization: a survey on principles, enabling technologies, and solutions. IEEE Commun. Surv. Tutor. 20(3), 2429\u20132453 (2018)","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"9631_CR14","unstructured":"$$\\mu$$ONOS: the next-generation architecture for the Open Network Operating System Controller. 2021. https:\/\/docs.onosproject.org\/"},{"key":"9631_CR15","unstructured":"ETSI NFV: NFV Technology. 2021. https:\/\/www.etsi.org\/technologies\/nfv"},{"key":"9631_CR16","doi-asserted-by":"crossref","unstructured":"Leonardo, \u00c1., Caraguay, V., Lude\u00c3a-Gonz\u00e1lez, P.J., Vicente, R., Tandazo, T., Isabel, L., L\u00f3pez, B.: SDN\/NFV architecture for IoT networks. In: Proceedings of the 14th International Conference on Web Information Systems and Technologies V.1: ITSCO, pp. 425\u2013429, Sept. (2018)","DOI":"10.5220\/0007234804250429"},{"issue":"6","key":"9631_CR17","doi-asserted-by":"publisher","first-page":"1994","DOI":"10.1109\/JIOT.2017.2746186","volume":"4","author":"S Bera","year":"2017","unstructured":"Bera, S., Misra, S., Vasilakos, A.V.: Software-defined networking for internet of things: a survey. IEEE Internet Things J. 4(6), 1994\u20132008 (2017)","journal-title":"IEEE Internet Things J."},{"issue":"6","key":"9631_CR18","doi-asserted-by":"publisher","first-page":"1400","DOI":"10.1109\/TPDS.2018.2883438","volume":"31","author":"D Wu","year":"2020","unstructured":"Wu, D., Nie, X., Asmare, E., Arkhipov, D.I., Qin, Z., Li, R., McCann, J.A., Li, K.: Towards distributed SDN: mobility management and flow scheduling in software defined urban IoT. IEEE Trans. Parallel Distrib. Syst. 31(6), 1400\u20131418 (2020)","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"issue":"5","key":"9631_CR19","doi-asserted-by":"publisher","first-page":"4119","DOI":"10.1109\/JIOT.2018.2867255","volume":"5","author":"C Mouradian","year":"2018","unstructured":"Mouradian, C., Jahromi, N.T., Glitho, R.H.: NFV and SDN-based distributed IoT gateway for large-scale disaster management. IEEE Internet Things J. 5(5), 4119\u20134131 (2018)","journal-title":"IEEE Internet Things J."},{"key":"9631_CR20","doi-asserted-by":"publisher","first-page":"229117","DOI":"10.1109\/ACCESS.2020.3045563","volume":"8","author":"J Okwuibe","year":"2020","unstructured":"Okwuibe, J., Haavisto, J., Harjula, E., Ahmad, I., Ylianttila, M.: SDN enhanced resource orchestration of containerized edge applications for industrial IoT. IEEE Access 8, 229117\u2013229131 (2020). https:\/\/doi.org\/10.1109\/ACCESS.2020.3045563","journal-title":"IEEE Access"},{"key":"9631_CR21","doi-asserted-by":"crossref","unstructured":"Villamizar, M., Garc\u00e9s, O., Castro, H., Verano, M., Salamanca, L., Casallas, R., Gil, S.: Evaluating the monolithic and the microservice architecture pattern to deploy web applications in the cloud. In: 2015 10th Computing Colombian Conference (10CCC), pp. 583\u2013590 (2015)","DOI":"10.1109\/ColumbianCC.2015.7333476"},{"key":"9631_CR22","doi-asserted-by":"publisher","DOI":"10.1109\/TNSM.2021.305975","author":"ST Arzo","year":"2021","unstructured":"Arzo, S.T., Bassoli, R., Granelli, F., Fitzek, F.H.P.: Multi-agent based autonomic network management architecture. IEEE Trans. Netw. Serv. Manag. (2021). https:\/\/doi.org\/10.1109\/TNSM.2021.305975","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"9631_CR23","doi-asserted-by":"publisher","unstructured":"Kookarinrat, P., Temtanapat, Y.: Design and implementation of a decentralized message bus for microservices. In: 2016 13th International Joint Conference on Computer Science and Software Engineering (JCSSE), pp. 1\u20136 (2016). https:\/\/doi.org\/10.1109\/JCSSE.2016.7748869.","DOI":"10.1109\/JCSSE.2016.7748869."},{"key":"9631_CR24","doi-asserted-by":"publisher","unstructured":"Chamas, C.L., Cordeiro, D., Eler, M.M.: Comparing REST, SOAP, Socket and gRPC in computation offloading of mobile applications: an energy cost analysis. In: 2017 IEEE 9th Latin-American Conference on Communications (LATINCOM), pp. 1\u20136 (2017). https:\/\/doi.org\/10.1109\/LATINCOM.2017.8240185.","DOI":"10.1109\/LATINCOM.2017.8240185."},{"key":"9631_CR25","unstructured":"3GPP: 3GPP TR 21.915 version 15.0.0 Release 15. 2021. https:\/\/www.etsi.org\/deliver\/etsi_tr\/121900_121999\/121915\/15.00.00_60\/tr_121915v150000p.pdf"},{"issue":"6","key":"9631_CR26","doi-asserted-by":"publisher","first-page":"1299","DOI":"10.1109\/LCOMM.2020.2979072","volume":"24","author":"T Das","year":"2020","unstructured":"Das, T., Gurusamy, M.: Controller placement for resilient network state synchronization in multi-controller SDN. IEEE Commun. Lett. 24(6), 1299\u20131303 (2020). https:\/\/doi.org\/10.1109\/LCOMM.2020.2979072","journal-title":"IEEE Commun. Lett."},{"key":"9631_CR27","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1145\/2875951.2875957","volume":"46","author":"L Schiff","year":"2016","unstructured":"Schiff, L., Schmid, S., Kuznetsov, P.: In-band synchronization for distributed SDN control planes. SIGCOMM Comput. Commun. Rev. 46, 37\u201343 (2016). https:\/\/doi.org\/10.1145\/2875951.2875957","journal-title":"SIGCOMM Comput. Commun. Rev."},{"issue":"4","key":"9631_CR28","doi-asserted-by":"publisher","first-page":"2926","DOI":"10.1109\/COMST.2016.2590438","volume":"18","author":"Martin L\u00e9vesque","year":"2016","unstructured":"L\u00e9vesque, Martin, Tipper, David: A survey of clock synchronization over packet-switched networks. IEEE Commun. Surv. Tutor. 18(4), 2926\u20132947 (2016). https:\/\/doi.org\/10.1109\/COMST.2016.2590438","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"9631_CR29","unstructured":"IEEE Std 1588-2019 (Revision of IEEE Std 1588-2008), IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems (2020)"},{"key":"9631_CR30","doi-asserted-by":"publisher","unstructured":"Kliazovich, D., Arzo, S.T., Granelli, F., Bouvry, P., Khan, S.U.: e-STAB: energy-efficient scheduling for cloud computing applications with traffic load balancing. In: 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing, pp. 7\u201313 (2013). https:\/\/doi.org\/10.1109\/GreenCom-iThings-CPSCom.2013.28.","DOI":"10.1109\/GreenCom-iThings-CPSCom.2013.28."},{"key":"9631_CR31","doi-asserted-by":"publisher","unstructured":"Gudipati, A., Perry, D., Li, L.E., Katti, S.: SoftRAN: software defined radio access network. In: Proceedings of the second ACM SIGCOMM workshop on Hot topics in software defined networking (HotSDN \u201913). Association for Computing Machinery, New York, NY, USA, 25\u201330 (2013). https:\/\/doi.org\/10.1145\/2491185.2491207","DOI":"10.1145\/2491185.2491207"},{"key":"9631_CR32","doi-asserted-by":"publisher","unstructured":"Raza, S.M., Kim, D.S., Choo, H.: The proposal for SDN supported future 5G networks. In: Proceedings of the 2014 Conference on Research in Adaptive and Convergent Systems (RACS \u201914). Association for Computing Machinery, New York, NY, USA, pp. 180\u2013185. https:\/\/doi.org\/10.1145\/2663761.2664237","DOI":"10.1145\/2663761.2664237"},{"issue":"2","key":"9631_CR33","doi-asserted-by":"publisher","first-page":"715","DOI":"10.1109\/TNSM.2019.2908675","volume":"16","author":"E Coronado","year":"2019","unstructured":"Coronado, E., Khan, S.N., Riggio, R.: 5G-EmPOWER: a software-defined networking platform for 5G radio access networks. IEEE Trans. Netw. Serv. Manag. 16(2), 715\u2013728 (2019). https:\/\/doi.org\/10.1109\/TNSM.2019.2908675","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"issue":"4","key":"9631_CR34","doi-asserted-by":"publisher","first-page":"2456","DOI":"10.1109\/COMST.2017.2736886","volume":"19","author":"A Gharaibeh","year":"2017","unstructured":"Gharaibeh, A., et al.: Smart cities: a survey on data management, security, and enabling technologies. IEEE Commun. Surv. Tutor. 19(4), 2456\u20132501 (2017). https:\/\/doi.org\/10.1109\/COMST.2017.2736886","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"9631_CR35","doi-asserted-by":"publisher","first-page":"1454","DOI":"10.1016\/j.jclepro.2016.10.006","volume":"140","author":"BLR Stojkoska","year":"2017","unstructured":"Stojkoska, B.L.R., Trivodaliev, K.V.: A review of Internet of Things for smart home: challenges and solutions. J. Clean. Prod. 140, 1454\u20131464 (2017). https:\/\/doi.org\/10.1016\/j.jclepro.2016.10.006","journal-title":"J. Clean. Prod."},{"issue":"7","key":"9631_CR36","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1109\/MC.2017.195","volume":"50","author":"MA Salahuddin","year":"2017","unstructured":"Salahuddin, M.A., Al-Fuqaha, A., Guizani, M., Shuaib, K., Sallabi, F.: Softwarization of Internet of Things infrastructure for secure and smart healthcare. Computer 50(7), 74\u201379 (2017). https:\/\/doi.org\/10.1109\/MC.2017.195","journal-title":"Computer"},{"key":"9631_CR37","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1016\/j.comnet.2017.04.038","volume":"121","author":"YE Oktian","year":"2017","unstructured":"Oktian, Y.E., Lee, S., Lee, H., Lam, J., Yustus Eko Oktian: Distributed SDN controller system: a survey on design choice. Comput. Netw. 121, 100\u2013111 (2017). https:\/\/doi.org\/10.1016\/j.comnet.2017.04.038","journal-title":"Comput. Netw."},{"key":"9631_CR38","unstructured":"Nicira Networks, NOX Controller. https:\/\/github.com\/noxrepo\/nox"},{"key":"9631_CR39","unstructured":"Open Source Development Platform, POX Controller. https:\/\/github.com\/noxrepo\/pox"},{"key":"9631_CR40","doi-asserted-by":"crossref","unstructured":"Bonafin, S., Bassoli, R., Granelli, F., Fitzek, F.H.P., Sacchi, C.: Virtual baseband unit splitting exploiting small satellite platforms In: 2020 IEEE Aerospace Conference, pp. 1\u201311 (2020)","DOI":"10.1109\/AERO47225.2020.9172316"},{"issue":"3","key":"9631_CR41","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1109\/LNET.2019.2918155","volume":"1","author":"D Comer","year":"2019","unstructured":"Comer, D., Rastegarnia, A.: Externalization of packet processing in software defined networking. IEEE Netw. Lett. 1(3), 124\u2013127 (2019)","journal-title":"IEEE Netw. Lett."},{"key":"9631_CR42","unstructured":"Comer, D., Rastegarnia, A.: Towards disaggregating the SDN control plane in CoRR (2019). arXiv:abs\/1902.00581"},{"key":"9631_CR43","unstructured":"P4: P4 Consortium. 2021. https:\/\/p4.org\/"},{"key":"9631_CR44","doi-asserted-by":"publisher","first-page":"134","DOI":"10.1016\/j.iot.2018.09.005","volume":"3\u20134","author":"L Bittencourt","year":"2018","unstructured":"Bittencourt, L., Immich, R., Sakellariou, R., Fonseca, N., Madeira, E., Curado, M., Villas, L., DaSilva, L., Lee, C., Rana, O.: The Internet of Things, Fog and Cloud continuum: integration and challenges. Internet Things 3\u20134, 134\u2013155 (2018). https:\/\/doi.org\/10.1016\/j.iot.2018.09.005","journal-title":"Internet Things"}],"container-title":["Journal of Network and Systems Management"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10922-021-09631-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10922-021-09631-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10922-021-09631-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,1,27]],"date-time":"2022-01-27T11:22:15Z","timestamp":1643282535000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10922-021-09631-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,21]]},"references-count":44,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1]]}},"alternative-id":["9631"],"URL":"https:\/\/doi.org\/10.1007\/s10922-021-09631-7","relation":{},"ISSN":["1064-7570","1573-7705"],"issn-type":[{"value":"1064-7570","type":"print"},{"value":"1573-7705","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,21]]},"assertion":[{"value":"2 April 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 September 2021","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 September 2021","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 October 2021","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"19"}}