{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,22]],"date-time":"2026-01-22T06:35:19Z","timestamp":1769063719793,"version":"3.49.0"},"reference-count":75,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2021,5,11]],"date-time":"2021-05-11T00:00:00Z","timestamp":1620691200000},"content-version":"vor","delay-in-days":130,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Wireless Communications and Mobile Computing"],"published-print":{"date-parts":[[2021,1]]},"abstract":"<jats:p>Developing specialized cloud\u2010based and open\u2010source testbeds is a practical approach to investigate network slicing functionalities in the fifth\u2010generation (5G) mobile networks. This paper provides a comprehensive review of most of the existing cost\u2010efficient and small\u2010scale testbeds that partially or fully deploy network slicing. First, we present relevant software packages for the three main functional blocks of the ETSI NFV MANO framework and for emulating the access and core network domains. Second, we define primary and secondary design criteria for deploying network slicing testbeds. These design criteria are later used for comparison between the testbeds. Third, we present the state\u2010of\u2010the\u2010art testbeds, including their design objectives, key technologies, network slicing deployment, and experiments. Next, we evaluate the testbeds according to the defined design criteria and present an in\u2010depth summary table. This assessment concludes with the superiority of some of them over the rest and the most dominant software packages satisfying the ETSI NFV MANO framework. Finally, challenges, potential solutions, and future works of network slicing testbeds are discussed.<\/jats:p>","DOI":"10.1155\/2021\/6655216","type":"journal-article","created":{"date-parts":[[2021,5,11]],"date-time":"2021-05-11T19:35:06Z","timestamp":1620761706000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["Small\u2010Scale 5G Testbeds for Network Slicing Deployment: A Systematic Review"],"prefix":"10.1155","volume":"2021","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3247-103X","authenticated-orcid":false,"given":"Ali","family":"Esmaeily","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1791-4434","authenticated-orcid":false,"given":"Katina","family":"Kralevska","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2021,5,11]]},"reference":[{"key":"e_1_2_9_1_2","doi-asserted-by":"publisher","DOI":"10.1109\/MWC.001.1900414"},{"key":"e_1_2_9_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2016.7470940"},{"key":"e_1_2_9_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMST.2015.2477041"},{"key":"e_1_2_9_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2017.1600935"},{"key":"e_1_2_9_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2014.2371999"},{"key":"e_1_2_9_6_2","doi-asserted-by":"publisher","DOI":"10.6028\/NIST.SP.800-145"},{"key":"e_1_2_9_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMST.2017.2745201"},{"key":"e_1_2_9_8_2","volume-title":"Network Functions Virtualisation (NFV): Architectural Framework","author":"ETSI","year":"2013"},{"key":"e_1_2_9_9_2","unstructured":"3GPP Study on Management and Orchestration of Network Slicing for Next Generation Network 2017."},{"key":"e_1_2_9_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.comnet.2020.107284"},{"key":"e_1_2_9_11_2","unstructured":"Stackify The Ultimate Guide to Performance Testing and Software Testing: Testing Types Performance Testing Steps Best Practices and More April 2021 https:\/\/stackify.com\/ultimate-guide-performance-testing-and-softwaretesting\/."},{"key":"e_1_2_9_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMST.2018.2815638"},{"key":"e_1_2_9_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2017.1600951"},{"key":"e_1_2_9_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMST.2016.2630047"},{"key":"e_1_2_9_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/access.2020.2975072"},{"key":"e_1_2_9_16_2","doi-asserted-by":"crossref","unstructured":"BonatiL. PoleseM. D\u2019OroS. BasagniS. andMelodiaT. Open programmable and virtualized 5G networks: state-of-the-art and the road ahead 2020 https:\/\/arxiv.org\/abs\/2005.10027.","DOI":"10.1016\/j.comnet.2020.107516"},{"key":"e_1_2_9_17_2","doi-asserted-by":"crossref","DOI":"10.1016\/j.comnet.2019.106984","article-title":"5G network slicing using SDN and NFV: a survey of taxonomy, architectures and future challenges","volume":"167","author":"Barakabitze A. A.","year":"2020","journal-title":"Computer Networks"},{"key":"e_1_2_9_18_2","unstructured":"OpenStack OpenStack The Most Widely Deployed Open Source Cloud Software in the World April 2021 https:\/\/www.openstack.org\/."},{"key":"e_1_2_9_19_2","unstructured":"OpenVIM Telef\u00f3nica NFV reference lab April 2021 https:\/\/github.com\/nfvlabs\/openvim."},{"key":"e_1_2_9_20_2","unstructured":"Kubernetes Kubernetes Production-Grade Container Orchestration April 2021 https:\/\/kubernetes.io."},{"key":"e_1_2_9_21_2","unstructured":"Open Source MANO (OSM) OSM Open Source NFV Management and Orchestration (MANO) software stack aligned with ETSI NFV April 2021 https:\/\/osm.etsi.org."},{"key":"e_1_2_9_22_2","unstructured":"ONAP ONAP Open Networking Automation Platform April 2021 https:\/\/www.onap.org\/."},{"key":"e_1_2_9_23_2","unstructured":"OpenBaton OpenBaton An extensible and customizable NFV MANO-compliant framework April 2021 http:\/\/openbaton.org."},{"key":"e_1_2_9_24_2","unstructured":"Cloudify Cloudify Multi Cloud Orchestration April 2021 https:\/\/cloudify.co\/."},{"key":"e_1_2_9_25_2","unstructured":"SONATA Sonata agile development. testing and orchestration of services in 5g virtualized networks April 2021 https:\/\/www.sonata-nfv.eu."},{"key":"e_1_2_9_26_2","unstructured":"Katana Wiki Home MediaNetworks Laboratory April 2021 https:\/\/github.com\/medianetlab\/katana-slice_manager\/wiki."},{"key":"e_1_2_9_27_2","doi-asserted-by":"crossref","unstructured":"Gomez-MiguelezI. Garcia-SaavedraA. SuttonP. SerranoP. CanoC. andLeithD. Srslte: An Opensource Platform for Lte Evolution and Experimentation 2016.","DOI":"10.1145\/2980159.2980163"},{"key":"e_1_2_9_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/2677046.2677053"},{"key":"e_1_2_9_29_2","unstructured":"O-RAN O-RAN Operator Defined Open and Intelligent Radio Access Networks April 2021 https:\/\/www.o-ran.org."},{"key":"e_1_2_9_30_2","volume-title":"Blog","author":"Murray J.","year":"2020"},{"key":"e_1_2_9_31_2","unstructured":"Open5GS Open5GS Open source project of 5GC and EPC April 2021 https:\/\/open5gs.org\/."},{"key":"e_1_2_9_32_2","volume-title":"OMEC Open Mobile Evolved Core","author":"TECHNICAL STEERING TEAM (TST)","year":"2020"},{"key":"e_1_2_9_33_2","unstructured":"free5GC free5GC Link the World! April 2021 https:\/\/www.free5gc.org\/."},{"key":"e_1_2_9_34_2","unstructured":"Open Networking Foundation (ONF) ODTN Open and Disaggregated Transport Network April 2021 https:\/\/www.opennetworking.org\/odtn\/."},{"key":"e_1_2_9_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2913185"},{"key":"e_1_2_9_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/2829988.2790041"},{"key":"e_1_2_9_37_2","doi-asserted-by":"crossref","unstructured":"KilincC. EricsonM. RugelandP. Da SilvaI. ZaidiA. AydinO. VenkatasubramanianV. FilippouM. C. MezzavillaM. KuruvattiN. andMonserratJ. F. 5G multi-rat integration evaluations using a common PDCP layer 2017 IEEE 85th Vehicular Technology Conference (VTC Spring) 2017 Sydney NSW Australia 1\u20135.","DOI":"10.1109\/VTCSpring.2017.8108594"},{"key":"e_1_2_9_38_2","doi-asserted-by":"crossref","unstructured":"DzogovicB. DoV. T. FengB. andvan DoT. Building virtualized 5G networks using open source software 2018 IEEE Symposium on Computer Applications Industrial Electronics (ISCAIE) 2018 Penang Malaysia 360\u2013366.","DOI":"10.1109\/ISCAIE.2018.8405499"},{"key":"e_1_2_9_39_2","doi-asserted-by":"crossref","unstructured":"DzogovicB. SantosB. DoV. T. FengB. JacotN. andVan DoT. Connecting remote eNodeB with containerized 5G C-RANs in OpenStack cloud 2019 6th IEEE International Conference on Cyber Security and Cloud Computing (CSCloud)\/ 2019 5th IEEE International Conference on Edge Computing and Scalable Cloud (EdgeCom) 2019 Paris France 14\u201319.","DOI":"10.1109\/CSCloud\/EdgeCom.2019.00013"},{"key":"e_1_2_9_40_2","doi-asserted-by":"crossref","unstructured":"ArifM. LiinamaaO. AhmadI. PouttuA. andYlianttilaM. On the demonstration and evaluation of service-based slices in 5G test network using NFV 2019 IEEE Wireless Communications and Networking Conference Workshop (WCNCW) 2019 Marrakech Morocco 1\u20136.","DOI":"10.1109\/WCNCW.2019.8902745"},{"key":"e_1_2_9_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2019.2917914"},{"key":"e_1_2_9_42_2","unstructured":"SEMIoTICS SEMIoTICS Smart End-to-end Massive IoT Interoperability Connectivity and Security April 2021 https:\/\/www.semiotics-project.eu\/."},{"key":"e_1_2_9_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3276799.3276803"},{"key":"e_1_2_9_44_2","doi-asserted-by":"crossref","unstructured":"FoukasX. NikaeinN. KassemM. M. MarinaM. K. andKontovasilisK. Flexran: a flexible and programmable platform for software-defined radio access networks Proceedings of the 12th International on Conference on Emerging Networking EXperiments and Technologies ser. CoNEXT \u201816 2016 Irvine California USA 427\u2013441 https:\/\/doi.org\/10.1145\/2999572.2999599 2-s2.0-85009754357.","DOI":"10.1145\/2999572.2999599"},{"key":"e_1_2_9_45_2","doi-asserted-by":"crossref","unstructured":"FoukasX. MarinaM. K. andKontovasilisK. P. Orion: Ran slicing for a flexible and cost-effective multiservice mobile network architecture In Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking (MobiCom \u201917). Association for Computing Machinery 2017 New York NY USA 127\u2013140 https:\/\/doi.org\/10.1145\/3117811.3117833 2-s2.0-85029724728.","DOI":"10.1145\/3117811.3117833"},{"key":"e_1_2_9_46_2","doi-asserted-by":"crossref","unstructured":"FoukasX. SardisF. FosterF. MarinaM. K. LemaM. A. andDohlerM. Experience building a prototype 5G testbed Proceedings of the Workshop on Experimentation and Measurements in 5G (EM-5G\u201918). Association for Computing Machinery 2018 New York NY USA 13\u201318 https:\/\/doi.org\/10.1145\/3286680.3286683 2-s2.0-85061792647.","DOI":"10.1145\/3286680.3286683"},{"key":"e_1_2_9_47_2","doi-asserted-by":"crossref","unstructured":"ShorovA. 5G testbed development for network slicing evaluation 2019 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus) 2019 Saint Petersburg and Moscow Russia 39\u201344.","DOI":"10.1109\/EIConRus.2019.8656861"},{"key":"e_1_2_9_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/MWC.2018.1800050"},{"key":"e_1_2_9_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.comcom.2019.11.003"},{"key":"e_1_2_9_50_2","doi-asserted-by":"publisher","DOI":"10.1155\/2019\/2361352"},{"key":"e_1_2_9_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNSM.2019.2908675"},{"key":"e_1_2_9_52_2","doi-asserted-by":"crossref","unstructured":"HuangC. HoC. NikaeinN. andChengR. Design and prototype of a virtualized 5G infrastructure supporting network slicing 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP) 2018 Shanghai China 1\u20135.","DOI":"10.1109\/ICDSP.2018.8631816"},{"key":"e_1_2_9_53_2","doi-asserted-by":"crossref","unstructured":"AbbasM. T. KhanT. A. MahmoodA. RiveraJ. J. D. andSongW. Introducing network slice management inside m-cord-based-5G framework NOMS 2018-2018 IEEE\/IFIP Network Operations and Management Symposium 2018 Taipei Taiwan 1\u20132.","DOI":"10.1109\/NOMS.2018.8406113"},{"key":"e_1_2_9_54_2","doi-asserted-by":"crossref","unstructured":"BerdeP. GerolaM. HartJ. HiguchiY. KobayashiM. KoideT. LantzB. O\u2019ConnorB. RadoslavovP. SnowW. andParulkarG. Onos: towards an open distributed SDN OS Proceedings of the Third Workshop on Hot Topics in Software Defined Networking ser. HotSDN\u201914 2014 Chicago IL USA 1\u20136 https:\/\/doi.org\/10.1145\/2620728.2620744 2-s2.0-84907015807.","DOI":"10.1145\/2620728.2620744"},{"key":"e_1_2_9_55_2","doi-asserted-by":"crossref","unstructured":"CostanzoS. FajjariI. AitsaadiN. andLangarR. Dynamic network slicing for 5G IoT and eMBB services: a new design with prototype and implementation results 2018 3rd Cloudification of the Internet of Things (CIoT) 2018 Paris France 1\u20137.","DOI":"10.1109\/CIOT.2018.8627115"},{"key":"e_1_2_9_56_2","doi-asserted-by":"crossref","unstructured":"FreitasL. A. BragaV. G. Corr\u00eaaS. L. MamatasL. RothenbergC. E. ClaymanS. andCardosoK. V. Slicing and allocation of transformable resources for the deployment of multiple virtualized infrastructure managers (vims) 4th IEEE Conference on Network Softwarization and Workshops (NetSoft) 2018 Montreal QC Canada 424\u2013432.","DOI":"10.1109\/NETSOFT.2018.8459990"},{"key":"e_1_2_9_57_2","unstructured":"ClaymanS. Network slicing supported by dynamic vim instantatiation IETF 100 2017 Singapore."},{"key":"e_1_2_9_58_2","doi-asserted-by":"crossref","unstructured":"SattarD.andMatrawyA. Dsaf: dynamic slice allocation framework for 5G core network 2019 https:\/\/arxiv.org\/abs\/1905.03873.","DOI":"10.1109\/ISNCC49221.2020.9297185"},{"key":"e_1_2_9_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/TBC.2019.2901402"},{"key":"e_1_2_9_60_2","first-page":"791","volume-title":"Interactive Mobile Communication, Technologies and Learning","author":"Ramantas K."},{"key":"e_1_2_9_61_2","doi-asserted-by":"crossref","unstructured":"KimJ.andXieM. A study of slice-aware service assurance for network function virtualization 2019 IEEE Conference on Network Softwarization (NetSoft) 2019 Paris France 489\u2013497.","DOI":"10.1109\/NETSOFT.2019.8806679"},{"key":"e_1_2_9_62_2","doi-asserted-by":"crossref","unstructured":"DreibholzT. Flexible 4G\/5G testbed setup for mobile edge computing using OpenAirInterface and open source mano 2020 Springer International Publishing Web Artificial Intelligence and Network Applications.","DOI":"10.1007\/978-3-030-44038-1_105"},{"key":"e_1_2_9_63_2","volume-title":"Integrating Cloud-RAN with Packet Core as VNF Using Open Source MANO and OpenAirInterface","author":"Ocampo A. F.","year":"2020"},{"key":"e_1_2_9_64_2","doi-asserted-by":"crossref","unstructured":"EsmaeilyA. KralevskaK. andGligoroskiD. A cloud-based SDN\/NFV testbed for end-to-end network slicing in 4G\/5G 2020 6th IEEE Conference on Network Softwarization (NetSoft) 2020 Ghent Belgium 29\u201335.","DOI":"10.1109\/NetSoft48620.2020.9165419"},{"key":"e_1_2_9_65_2","doi-asserted-by":"crossref","unstructured":"HagaS. EsmaeilyA. KralevskaK. andGligoroskiD. 5G network slice isolation with WireGuard and open source MANO: a VPNaaS Proof-of-Concept 2020 IEEE Conference on Network Function Virtualization and Software Defined Networks NFV-SDN 2020 2020 Leganes-Madrid Spain 181\u2013187 https:\/\/doi.org\/10.1109\/NFV-SDN50289.2020.9289900.","DOI":"10.1109\/NFV-SDN50289.2020.9289900"},{"key":"e_1_2_9_66_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2017.23160"},{"key":"e_1_2_9_67_2","doi-asserted-by":"crossref","unstructured":"RodriguezV. Q. GuilleminF. andBoubendirA. 5G e2e network slicing management with onap 2020 23rd Conference on Innovation in Clouds Internet and Networks and Workshops (ICIN) 2020 Paris France 87\u201394.","DOI":"10.1109\/ICIN48450.2020.9059507"},{"key":"e_1_2_9_68_2","doi-asserted-by":"publisher","DOI":"10.12720\/jcm.14.12.1110-1118"},{"key":"e_1_2_9_69_2","doi-asserted-by":"crossref","unstructured":"SarrigiannisI. KartsakliE. RamantasK. AntonopoulosA. andVerikoukisC. Application and network VNF migration in a MEC-enabled 5G architecture 2018 IEEE 23rd International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD) 2018 Barcelona Spain 1\u20136 https:\/\/doi.org\/10.1109\/CAMAD.2018.8514943 2-s2.0-85057285501.","DOI":"10.1109\/CAMAD.2018.8514943"},{"key":"e_1_2_9_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2019.2944695"},{"key":"e_1_2_9_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3043876"},{"key":"e_1_2_9_72_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4842-4470-8_46"},{"key":"e_1_2_9_73_2","unstructured":"Prometheus Prometheus From metrics to insight April 2021 https:\/\/prometheus.io\/."},{"key":"e_1_2_9_74_2","unstructured":"Grafana Grafana Labs Your observability wherever you need it April 2021 https:\/\/grafana.com\/."},{"key":"e_1_2_9_75_2","volume-title":"Osm Release Eight Notes","author":"Technical Steering Committee","year":"2020"}],"container-title":["Wireless Communications and Mobile Computing"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/wcmc\/2021\/6655216.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/wcmc\/2021\/6655216.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/2021\/6655216","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T11:00:43Z","timestamp":1723028443000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1155\/2021\/6655216"}},"subtitle":[],"editor":[{"given":"Luis","family":"Castedo","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2021,1]]},"references-count":75,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,1]]}},"alternative-id":["10.1155\/2021\/6655216"],"URL":"https:\/\/doi.org\/10.1155\/2021\/6655216","archive":["Portico"],"relation":{},"ISSN":["1530-8669","1530-8677"],"issn-type":[{"value":"1530-8669","type":"print"},{"value":"1530-8677","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1]]},"assertion":[{"value":"2020-11-04","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-04-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-05-11","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"6655216"}}