{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T11:21:28Z","timestamp":1778498488858,"version":"3.51.4"},"reference-count":69,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2022,1,17]],"date-time":"2022-01-17T00:00:00Z","timestamp":1642377600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100008725","name":"Agencia Nacional de Investigaci\u00f3n e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["POS_NAC_M_2020_1_163847"],"award-info":[{"award-number":["POS_NAC_M_2020_1_163847"]}],"id":[{"id":"10.13039\/100008725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>The Internet architecture has been undergoing a significant refactoring, where the past preeminence of transit providers has been replaced by content providers, which have a ubiquitous presence throughout the world, seeking to improve the user experience, bringing content closer to its final recipients. This restructuring is materialized in the emergence of Massive Scale Data Centers (MSDC) worldwide, which allows the implementation of the Cloud Computing concept. MSDC usually deploy Fat-Tree topologies, with constant bisection bandwidth among servers and multi-path routing. To take full advantage of such characteristics, specific routing protocols are needed. Multi-path routing also calls for revision of transport protocols and forwarding policies, also affected by specific MSDC applications\u2019 traffic characteristics. Experimenting over these infrastructures is prohibitively expensive, and therefore, scalable and realistic experimentation environments are needed to research and test solutions for MSDC. In this paper, we review several environments, both single-host and distributed, which permit analyzing the pros and cons of different solutions.<\/jats:p>","DOI":"10.3390\/fi14010029","type":"journal-article","created":{"date-parts":[[2022,1,17]],"date-time":"2022-01-17T08:20:42Z","timestamp":1642407642000},"page":"29","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Experimentation Environments for Data Center Routing Protocols: A Comprehensive Review"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8228-1648","authenticated-orcid":false,"given":"Leonardo","family":"Alberro","sequence":"first","affiliation":[{"name":"Instituto de Computaci\u00f3n (InCo), Universidad de la Rep\u00fablica (UdelaR), Montevideo 11300, Uruguay"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9174-398X","authenticated-orcid":false,"given":"Alberto","family":"Castro","sequence":"additional","affiliation":[{"name":"Instituto de Computaci\u00f3n (InCo), Universidad de la Rep\u00fablica (UdelaR), Montevideo 11300, Uruguay"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6046-0023","authenticated-orcid":false,"given":"Eduardo","family":"Grampin","sequence":"additional","affiliation":[{"name":"Instituto de Computaci\u00f3n (InCo), Universidad de la Rep\u00fablica (UdelaR), Montevideo 11300, Uruguay"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,17]]},"reference":[{"key":"ref_1","unstructured":"(2021, December 12). The Death of Transit?|APNIC Blog. Available online: https:\/\/blog.apnic.net\/2016\/10\/28\/the-death-of-transit\/."},{"key":"ref_2","unstructured":"Cisco (2021, December 12). Cisco Global Cloud Index: Forecast and Methodology, 2016\u20132021, Available online: https:\/\/virtualization.network\/Resources\/Whitepapers\/0b75cf2e-0c53-4891-918e-b542a5d364c5_white-paper-c11-738085.pdf."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"406","DOI":"10.1002\/j.1538-7305.1953.tb01433.x","article-title":"A study of non-blocking switching networks","volume":"32","author":"Clos","year":"1953","journal-title":"Bell Syst. Tech. J."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Al-Fares, M., Loukissas, A., and Vahdat, A. (2008, January 17\u201322). A Scalable, Commodity Data Center Network Architecture. Proceedings of the ACM SIGCOMM 2008 Conference on Data Communication, Seattle, WA, USA.","DOI":"10.1145\/1402958.1402967"},{"key":"ref_5","unstructured":"Medhi, D., and Ramasamy, K. (2017). Network Routing, Second Edition: Algorithms, Protocols, and Architectures, Morgan Kaufmann Publishers Inc.. [2nd ed.]."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"909","DOI":"10.1109\/SURV.2012.090512.00043","article-title":"Data Center Network Virtualization: A Survey","volume":"15","author":"Bari","year":"2013","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Handley, M., Raiciu, C., Agache, A., Voinescu, A., Moore, A.W., Antichi, G., and W\u00f3jcik, M. (2017, January 21\u201325). Re-Architecting Datacenter Networks and Stacks for Low Latency and High Performance. Proceedings of the Conference of the ACM Special Interest Group on Data Communication, Los Angeles, CA, USA.","DOI":"10.1145\/3098822.3098825"},{"key":"ref_8","unstructured":"RFC (2021, December 12). Use of BGP for Routing in Large-Scale Data Centers, Available online: https:\/\/tools.ietf.org\/html\/rfc7938."},{"key":"ref_9","unstructured":"White, R., Hegde, S., and Zandi, S. (2021, December 12). IS-IS Optimal Distributed Flooding for Dense Topologies. Internet-Draft Draft-White-Distoptflood-00, IETF Secretariat. Available online: https:\/\/datatracker.ietf.org\/doc\/html\/draft-white-lsr-distoptflood-00."},{"key":"ref_10","unstructured":"Przygienda, T., Sharma, A., Thubert, P., Rijsman, B., and Afanasiev, D. (2021, December 12). RIFT: Routing in Fat Trees. Internet-Draft Draft-Ietf-Rift-Rift-12, IETF Secretariat. Available online: https:\/\/datatracker.ietf.org\/doc\/html\/draft-ietf-rift-rift-12."},{"key":"ref_11","unstructured":"Aelmans, M., Vandezande, O., Rijsman, B., Head, J., Graf, C., Alberro, L., Mali, H., and Steudler, O. (2021, December 12). Day One: Routing in Fat Trees (RIFT), Available online: https:\/\/www.juniper.net\/documentation\/en_US\/day-one-books\/DO_RIFT.pdf."},{"key":"ref_12","unstructured":"Patel, K., Lindem, A., Zandi, S., and Henderickx, W. (2021, December 12). BGP Link-State Shortest Path First (SPF) Routing. Internet-Draft Draft-Ietf-Lsvr-Bgp-Spf-13, IETF Secretariat. Available online: https:\/\/www.ietf.org\/archive\/id\/draft-ietf-lsvr-bgp-spf-13.txt."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"892","DOI":"10.1109\/TC.1985.6312192","article-title":"Fat-trees: Universal networks for hardware-efficient supercomputing","volume":"C-34","author":"Leiserson","year":"1985","journal-title":"IEEE Trans. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Moy, J. (1998). OSPF Version 2, IETF. RFC 2328.","DOI":"10.17487\/rfc2328"},{"key":"ref_15","unstructured":"Shen, N., Ginsberg, L., and Thyamagundalu, S. (2021, December 12). IS-IS Routing for Spine-Leaf Topology. Internet-Draft Draft-Shen-Isis-Spine-Leaf-Ext-07, IETF Secretariat. Available online: https:\/\/datatracker.ietf.org\/doc\/html\/draft-shen-isis-spine-leaf-ext-07."},{"key":"ref_16","unstructured":"Martey, A., Sturgess, S., and Martey, A. (2021, December 12). IS-IS Network Design Solutions, Available online: https:\/\/www.ciscopress.com\/store\/is-is-network-design-solutions-9781578702206."},{"key":"ref_17","unstructured":"Zandi, S. (2021, December 12). LinkedIn OpenFabric Project. Available online: https:\/\/www.slideshare.net\/shawnzandi\/linkedin-openfabric-project-interop-2017."},{"key":"ref_18","unstructured":"Rekhter, Y., Li, T., and Hares, S. (2021, December 12). A Border Gateway Protocol 4 (BGP-4). RFC 4271, RFC Editor. Available online: http:\/\/www.rfc-editor.org\/rfc\/rfc4271.txt."},{"key":"ref_19","unstructured":"RFC (2021, December 12). BGP Support for Four-Octet AS Number Space, Available online: https:\/\/datatracker.ietf.org\/doc\/html\/rfc4893."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1109\/TNET.2009.2016390","article-title":"Quantifying Path Exploration in the Internet","volume":"17","author":"Oliveira","year":"2009","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_21","unstructured":"Dutt, D.G. (2017). BGP in the Data Center, O\u2019Reilly."},{"key":"ref_22","unstructured":"RFC (2021, December 12). BGP\u2014Link State (BGP-LS) Advertisement of IGP Traffic Engineering Performance Metric Extensions, Available online: https:\/\/datatracker.ietf.org\/doc\/html\/rfc8571."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger, L., Sridhar, T., Bursell, M., and Wright, C. (2014). Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks, IETF. RFC 7348.","DOI":"10.17487\/rfc7348"},{"key":"ref_24","unstructured":"Sajassi, A., Drake, J., Bitar, N., Shekhar, R., Uttaro, J., and Henderickx, W. (2018). A Network Virtualization Overlay Solution Using Ethernet VPN (EVPN), IETF. RFC 8365."},{"key":"ref_25","unstructured":"Roman (2021, December 12). Nokia SR Linux Goes Public. Available online: https:\/\/netdevops.me\/2021\/nokia-sr-linux-goes-public\/."},{"key":"ref_26","unstructured":"Cisco (2021, December 12). NetDevOps-Cisco DevNet. Available online: https:\/\/developer.cisco.com\/netdevops\/."},{"key":"ref_27","unstructured":"Maor, I. (2021, December 12). Make a Digital Twin of Your Data Center. Available online: https:\/\/developer.nvidia.com\/blog\/make-a-digital-twin-of-your-data-center-with-sonic-running-on-nvidia-air\/."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Liu, H., Zhu, Y., Padhye, J., Cao, J., Tallapragada, S., Lopes, N., Rybalchenko, A., Lu, G., and Yuan, L. (2017, January 28\u201331). CrystalNet: Faithfully Emulating Large Production Networks. Proceedings of the 26th Symposium on Operating Systems Principles, Shanghai, China.","DOI":"10.1145\/3132747.3132759"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Hong, H., Wu, Q., Dong, F., Song, W., Sun, R., Han, T., Zhou, C., and Yang, H. (2021, January 23). NetGraph: An Intelligent Operated Digital Twin Platform for Data Center Networks. Proceedings of the ACM SIGCOMM 2021 Workshop on Network-Application Integration, Virtual.","DOI":"10.1145\/3472727.3472802"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"940","DOI":"10.1109\/COMST.2018.2868050","article-title":"A Survey on Network Verification and Testing With Formal Methods: Approaches and Challenges","volume":"21","author":"Li","year":"2019","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Meza, J., Xu, T., Veeraraghavan, K., and Mutlu, O. (November, January 31). A Large Scale Study of Data Center Network Reliability. Proceedings of the Internet Measurement Conference 2018, Boston, MA, USA.","DOI":"10.1145\/3278532.3278566"},{"key":"ref_32","unstructured":"Nsnam (2021, December 12). ns-3 a Discrete-Event Network Simulator for Internet Systems. Available online: https:\/\/www.nsnam.org\/."},{"key":"ref_33","unstructured":"OpenSim Ltd (2021, December 12). OMNeT++ - Discrete Event Simulator. Available online: https:\/\/omnetpp.org\/."},{"key":"ref_34","unstructured":"FRRouting (2021, December 12). FRRouting. Available online: https:\/\/frrouting.org\/."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1109\/MNET.2018.1700277","article-title":"Assessing the Limits of Mininet-Based Environments for Network Experimentation","volume":"32","author":"Muelas","year":"2018","journal-title":"IEEE Netw."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Pediaditakis, D., Rotsos, C., and Moore, A.W. (2014, January 20\u201321). Faithful reproduction of network experiments. Proceedings of the 2014 ACM\/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), Marina del Rey, CA, USA.","DOI":"10.1145\/2658260.2658274"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Bonofiglio, G., Iovinella, V., Lospoto, G., and Di Battista, G. (2018, January 23\u201327). Kathar\u00e1: A container-based framework for implementing network function virtualization and software defined networks. Proceedings of the NOMS 2018\u20142018 IEEE\/IFIP Network Operations and Management Symposium, Taipei, Taiwan.","DOI":"10.1109\/NOMS.2018.8406267"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Scazzariello, M., Ariemma, L., and Caiazzi, T. (2020, January 20\u201324). Kathar\u00e1: A Lightweight Network Emulation System. Proceedings of the NOMS 2020\u20142020 IEEE\/IFIP Network Operations and Management Symposium, Budapest, Hungary.","DOI":"10.1109\/NOMS47738.2020.9110351"},{"key":"ref_39","first-page":"2","article-title":"Docker: Lightweight linux containers for consistent development and deployment","volume":"2014","author":"Merkel","year":"2014","journal-title":"Linux J."},{"key":"ref_40","unstructured":"Quagga (2021, December 12). Quagga Routing Suite. Available online: https:\/\/www.quagga.net\/."},{"key":"ref_41","unstructured":"Lumbis, P. (2021, December 12). FRR: The Most Popular Network Router You\u2019ve Never Heard of. Available online: https:\/\/www.nextplatform.com\/2020\/10\/26\/frr-the-most-popular-network-router-youve-never-heard-of\/."},{"key":"ref_42","unstructured":"Kathar\u00e1 (2021, December 12). Kathar\u00e1. Available online: https:\/\/github.com\/KatharaFramework\/Kathara."},{"key":"ref_43","unstructured":"Kubernetes (2021, December 12). Kubernetes. Available online: https:\/\/kubernetes.io\/."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Scazzariello, M., Ariemma, L., Battista, G.D., and Patrignani, M. (2020, January 20\u201324). Megalos: A Scalable Architecture for the Virtualization of Network Scenarios. Proceedings of the NOMS 2020\u20142020 IEEE\/IFIP Network Operations and Management Symposium, Budapest, Hungary.","DOI":"10.1109\/NOMS47738.2020.9110288"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Caiazzi, T., Scazzariello, M., Alberro, L., Ariemma, L., Castro, A., Grampin, E., and Battista, G.D. (2021, December 12). Sibyl: A Framework for Evaluating the Implementation of Routing Protocols in Fat-Trees. Available online: https:\/\/gitlab.com\/uniroma3\/compunet\/networks\/sibyl-framework\/sibyl.","DOI":"10.1109\/NOMS54207.2022.9789876"},{"key":"ref_46","unstructured":"Caiazzi, T., Scazzariello, M., and Ariemma, L. (2021, January 17\u201321). VFTGen: A Tool to Perform Experiments in Virtual Fat Tree Topologies. Proceedings of the IM 2021\u20142021 IFIP\/IEEE International Symposium on Integrated Network Management, Bordeaux, France."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Lantz, B., Heller, B., and McKeown, N. (2010, January 20\u201321). A Network in a Laptop: Rapid Prototyping for Software-Defined Networks. Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks, Monterey, CA, USA.","DOI":"10.1145\/1868447.1868466"},{"key":"ref_48","unstructured":"Team, M. (2021, December 12). Mininet an Instant Virtual Network on Your Laptop (or Other PC). Available online: http:\/\/mininet.org\/."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Ruslan, R., Othman, N.B., Fuzi, M.M., and Ghazali, N. (2020, January 21\u201322). Scalability Analysis in Mininet on Software Defined Network using ONOS. Proceedings of the 2020 Emerging Technology in Computing, Communication and Electronics (ETCCE), Dhaka, Bangladesh.","DOI":"10.1109\/ETCCE51779.2020.9350892"},{"key":"ref_50","unstructured":"Open Networking Foundation (2021, December 12). Open Network Operating System (ONOS). Available online: https:\/\/opennetworking.org\/onos\/."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Ahrenholz, J., Danilov, C., Henderson, T.R., and Kim, J.H. (2008, January 16\u201319). CORE: A real-time network emulator. Proceedings of the MILCOM 2008\u20142008 IEEE Military Communications Conference, San Diego, CA, USA.","DOI":"10.1109\/MILCOM.2008.4753614"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Ahrenholz, J. (November, January 31). Comparison of CORE network emulation platforms. Proceedings of the MILCOM 2010 Military Communications Conference, San Jose, CA, USA.","DOI":"10.1109\/MILCOM.2010.5680218"},{"key":"ref_53","unstructured":"Technologies, G. (2021, December 12). GNS3: The Software That Empowers Network Professionals. Available online: https:\/\/www.gns3.com\/."},{"key":"ref_54","unstructured":"Orebaugh, A., Ramirez, G., Beale, J., and Wright, J. (2007). Wireshark &amp, Syngress Publishing. Ethereal Network Protocol Analyzer Toolkit."},{"key":"ref_55","unstructured":"Tcpdump (2021, December 12). Tcpdump. Available online: https:\/\/www.tcpdump.org\/."},{"key":"ref_56","unstructured":"Wireshark (2021, December 12). TSHARK Manual Page. Available online: https:\/\/www.wireshark.org\/docs\/man-pages\/tshark.html."},{"key":"ref_57","unstructured":"KimiNewt (2021, December 12). Pyshark. Available online: https:\/\/github.com\/KimiNewt\/pyshark."},{"key":"ref_58","first-page":"200892","article-title":"Packet analysis for network forensics: A comprehensive survey","volume":"32","author":"Sikos","year":"2020","journal-title":"Forensic Sci. Int. Digit. Investig."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Chapman, C. (2016). Chapter 7\u2014Using Wireshark and TCP dump to visualize traffic. Network Performance and Security, Syngress.","DOI":"10.1016\/B978-0-12-803584-9.00007-X"},{"key":"ref_60","unstructured":"Rijsman, B. (2021, December 12). Routing in Fat Trees (RIFT). Available online: https:\/\/github.com\/brunorijsman\/rift-python."},{"key":"ref_61","unstructured":"Panandr (2021, December 12). Mininet-Fattree. Available online: https:\/\/github.com\/panandr\/mininet-fattree."},{"key":"ref_62","unstructured":"Mininet (2021, December 12). Mininet Python API Reference Manual. Available online: http:\/\/mininet.org\/api\/annotated."},{"key":"ref_63","unstructured":"Common Open Research Emulator (2021, December 12). Python API. Available online: http:\/\/coreemu.github.io\/core\/python."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Wette, P., Dr\u00e4xler, M., Schwabe, A., Wallaschek, F., Zahraee, M.H., and Karl, H. (2014, January 2\u20134). MaxiNet: Distributed emulation of software-defined networks. Proceedings of the 2014 IFIP Networking Conference, Trondheim, Norway.","DOI":"10.1109\/IFIPNetworking.2014.6857078"},{"key":"ref_65","first-page":"3","article-title":"Distrinet: A mininet implementation for the cloud","volume":"51","author":"Tomassilli","year":"2021","journal-title":"Comput. Commun. Rev."},{"key":"ref_66","unstructured":"OpenWrt Project (2021, December 12). OpenWrt Project. Available online: https:\/\/openwrt.org\/."},{"key":"ref_67","unstructured":"Lucero, M., Parnizari, A., Alberro, L., Castro, A., and Gramp\u00edn, E. (2021, January 17\u201321). Routing in Fat Trees: A protocol analyzer for debugging and experimentation. Proceedings of the 2021 IFIP\/IEEE International Symposium on Integrated Network Management (IM), Bordeaux, France."},{"key":"ref_68","unstructured":"NetworkX (2021, December 12). NetworkX. Available online: https:\/\/networkx.org\/."},{"key":"ref_69","unstructured":"Alberro, L. (2021, December 12). Experimentation Environments for Routing Protocols in the Datacenter. Available online: https:\/\/gitlab.com\/fing-mina\/datacenters\/fat-tree-emulators."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/1\/29\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:02:36Z","timestamp":1760133756000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/1\/29"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,17]]},"references-count":69,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["fi14010029"],"URL":"https:\/\/doi.org\/10.3390\/fi14010029","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,17]]}}}