{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:20:36Z","timestamp":1760059236389,"version":"build-2065373602"},"reference-count":35,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2025,5,29]],"date-time":"2025-05-29T00:00:00Z","timestamp":1748476800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002347","name":"Federal Ministry of Education and Research","doi-asserted-by":"publisher","award":["01IS22093A","01IS22093B"],"award-info":[{"award-number":["01IS22093A","01IS22093B"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Kubernetes has emerged as the industry standard for container orchestration in cloud environments, with its scheduler dynamically placing container instances across cluster nodes based on predefined rules and algorithms. Various efforts have been made to extend and improve upon the Kubernetes scheduler. However, as the majority of Kubernetes clusters operate on homogeneous hardware, most scheduling algorithms are also only developed for homogeneous systems. Heterogeneous infrastructures, which include IoT devices or specialized hardware, have become more widespread and require specialized tuning to optimize workload assignment, for which researchers and developers working on scheduling systems require access to heterogeneous hardware for development and testing; such data may not be available. While simulations such as CloudSim or K8sSim can provide insights, the level of detail they can offer to validate new schedulers is limited, as they are only simulations. To address this, we introduce Q8S, a tool for emulating heterogeneous Kubernetes clusters including x86_64 and ARM64 architectures on OpenStack using QEMU. Emulations created through Q8S provide a higher level of detail than simulations and can be used to train machine learning scheduling algorithms. By providing an environment capable of executing real workloads, Q8S enables researchers and developers to test and refine their scheduling algorithms, ultimately leading to more efficient and effective heterogeneous cluster management. We release our implementation of Q8S as open source.<\/jats:p>","DOI":"10.3390\/a18060324","type":"journal-article","created":{"date-parts":[[2025,5,29]],"date-time":"2025-05-29T04:46:38Z","timestamp":1748493998000},"page":"324","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Q8S: Emulation of Heterogeneous Kubernetes Clusters Using QEMU"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7384-7304","authenticated-orcid":false,"given":"Jonathan","family":"Decker","sequence":"first","affiliation":[{"name":"Institute for Computer Science, University of G\u00f6ttingen, Goldschmidtstra\u00dfe 7, 37077 G\u00f6ttingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-6640-4317","authenticated-orcid":false,"given":"Vincent Florens","family":"Hasse","sequence":"additional","affiliation":[{"name":"Institute for Computer Science, University of G\u00f6ttingen, Goldschmidtstra\u00dfe 7, 37077 G\u00f6ttingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6915-1179","authenticated-orcid":false,"given":"Julian","family":"Kunkel","sequence":"additional","affiliation":[{"name":"Institute for Computer Science, University of G\u00f6ttingen, Goldschmidtstra\u00dfe 7, 37077 G\u00f6ttingen, Germany"},{"name":"GWDG, Burckhardtweg 4, 37077 G\u00f6ttingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,5,29]]},"reference":[{"key":"ref_1","first-page":"3","article-title":"Cloud Computing and Its Applications: A Comprehensive Survey","volume":"28","author":"Kiswani","year":"2021","journal-title":"Int. J. Comput. Their Appl."},{"key":"ref_2","first-page":"53","article-title":"The Future of Cloud Computing: Benefits and Challenges","volume":"16","author":"Islam","year":"2023","journal-title":"Int. J. Commun. Netw. Syst. Sci."},{"key":"ref_3","unstructured":"(2024, September 21). CNCF Annual Survey 2023. Available online: https:\/\/www.cncf.io\/reports\/cncf-annual-survey-2023\/."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1186\/s13677-023-00471-1","article-title":"A Survey of Kubernetes Scheduling Algorithms","volume":"12","author":"Senjab","year":"2023","journal-title":"J. Cloud Comput."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.48161\/qaj.v1n2a36","article-title":"IoT and Cloud Computing Issues, Challenges and Opportunities: A Review","volume":"1","author":"Sadeeq","year":"2021","journal-title":"Qubahan Acad. J."},{"key":"ref_6","first-page":"138:1","article-title":"Kubernetes Scheduling: Taxonomy, Ongoing Issues and Challenges","volume":"55","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3934","DOI":"10.1016\/j.jksuci.2021.03.002","article-title":"Container Scheduling Techniques: A Survey and Assessment","volume":"34","author":"Ahmad","year":"2022","journal-title":"J. King Saud Univ.\u2014Comput. Inf. Sci."},{"key":"ref_8","unstructured":"Mars, J., Tang, L., and Hundt, R. (2024, September 21). Heterogeneity in \u201cHomogeneous\u201d Warehouse-Scale Computers: A Performance Opportunity. Available online: https:\/\/ieeexplore.ieee.org\/abstract\/document\/5887296."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Kunkel, J., Boehme, C., Decker, J., Magugliani, F., Pleiter, D., Koller, B., Sivalingam, K., Pllana, S., Nikolov, A., and Soyturk, M. (2023, January 9\u201311). DECICE: Device-edge-cloud Intelligent Collaboration Framework. Proceedings of the Computing Frontiers, Bologna, Italy.","DOI":"10.1145\/3587135.3592179"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1002\/spe.995","article-title":"CloudSim: A Toolkit for Modeling and Simulation of Cloud Computing Environments and Evaluation of Resource Provisioning Algorithms","volume":"41","author":"Calheiros","year":"2011","journal-title":"Softw. Pract. Exp."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Bux, M., and Leser, U. (2013, January 23). DynamicCloudSim: Simulating Heterogeneity in Computational Clouds. Proceedings of the 2nd ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies (SWEET \u201913), New York, NY, USA.","DOI":"10.1145\/2499896.2499897"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kathiravelu, P., and Veiga, L. (2014, January 9\u201311). Concurrent and Distributed CloudSim Simulations. Proceedings of the 2014 IEEE 22nd International Symposium on Modelling, Analysis & Simulation of Computer and Telecommunication Systems, Paris, France.","DOI":"10.1109\/MASCOTS.2014.70"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wu, J., Xu, M., He, Y., Ye, K., and Xu, C. (2025). Cloudnativesim: A Toolkit for Modeling and Simulation of Cloud-Native Applications. Softw. Pract. Exp., 1\u201324.","DOI":"10.1002\/spe.3417"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3959","DOI":"10.1007\/s11227-020-03425-5","article-title":"PerficientCloudSim: A Tool to Simulate Large-Scale Computation in Heterogeneous Clouds","volume":"77","author":"Zakarya","year":"2021","journal-title":"J. Supercomput."},{"key":"ref_15","unstructured":"Monti, F., Rinderle-Ma, S., Ruiz Cort\u00e9s, A., Zheng, Z., and Mecella, M. (December, January 28). ServiceSim: A Modelling and Simulation Toolkit of Microservice Systems in Cloud-Edge Environment. Proceedings of the Service-Oriented Computing, Rome, Italy."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1002\/spe.2124","article-title":"EMUSIM: An Integrated Emulation and Simulation Environment for Modeling, Evaluation, and Validation of Performance of Cloud Computing Applications","volume":"43","author":"Calheiros","year":"2013","journal-title":"Softw. Pract. Exp."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wen, S., Han, R., Qiu, K., Ma, X., Li, Z., Deng, H., and Liu, C.H. (2023). K8sSim: A Simulation Tool for Kubernetes Schedulers and Its Applications in Scheduling Algorithm Optimization. Micromachines, 14.","DOI":"10.3390\/mi14030651"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Akbari, N., Toosi, A.N., Grundy, J., Khalajzadeh, H., Aslanpour, M.S., and Ilager, S. (2024, January 7\u201313). iContinuum: An Emulation Toolkit for Intent-Based Computing Across the Edge-to-Cloud Continuum. Proceedings of the 2024 IEEE 17th International Conference on Cloud Computing (CLOUD), Shenzhen, China.","DOI":"10.1109\/CLOUD62652.2024.00059"},{"key":"ref_19","unstructured":"(2024, August 02). QEMU. Available online: https:\/\/wiki.qemu.org\/Main_Page."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zeng, Z., Chung, C.J., and Xie, L. (2024, January 24\u201327). The Development of A Large-Scale Cloud Emulator. Proceedings of the 2024 IEEE International Conference on Cloud Engineering (IC2E), Paphos, Cyprus.","DOI":"10.1109\/IC2E61754.2024.00030"},{"key":"ref_21","unstructured":"(2024, September 10). GitHub\u2014InfraBuilder\/K8s-Bench-Suite: Simple Scripts to Benchmark Kubernetes Cluster Features. Available online: https:\/\/github.com\/InfraBuilder\/k8s-bench-suite."},{"key":"ref_22","unstructured":"(2024, September 21). Libvirt: The Virtualization API. Available online: https:\/\/libvirt.org\/."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Anthony, R.J. (2016). Chapter 5\u2014The Architecture View. Systems Programming, Morgan Kaufmann.","DOI":"10.1016\/B978-0-12-800729-7.00005-4"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1016\/j.protcy.2012.03.029","article-title":"A Summary of Virtualization Techniques","volume":"3","author":"Freitag","year":"2012","journal-title":"Procedia Technol."},{"key":"ref_25","unstructured":"Song, Y., Wang, H., and Soyata, T. (2015). Hardware and Software Aspects of VM-Based Mobile-Cloud Offloading, IGI-Global."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tsetse, A., Tweneboah-Koduah, S., Rawal, B., Zheng, Z., and Prattipati, M. (August, January 30). A Comparative Study of System Virtualization Performance. Proceedings of the 2019 IEEE 20th International Conference on Information Reuse and Integration for Data Science (IRI), Los Angeles, CA, USA.","DOI":"10.1109\/IRI.2019.00064"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lim, J.T., and Nieh, J. (2020, January 16\u201320). Optimizing Nested Virtualization Performance Using Direct Virtual Hardware. Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, Lausanne, Switzerland.","DOI":"10.1145\/3373376.3378467"},{"key":"ref_28","first-page":"1","article-title":"Overview of Kubernetes CNI Plugins Performance","volume":"12","year":"2020","journal-title":"Moksl.\u2014Liet. Ateitis"},{"key":"ref_29","unstructured":"Bhatia, S.K., Tiwari, S., Ruidan, S., Trivedi, M.C., and Mishra, K.K. (2019, January 11\u201312). Networking Analysis and Performance Comparison of Kubernetes CNI Plugins. Proceedings of the Advances in Computer, Communication and Computational Sciences, Bangkok, Thailand."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Maria, A. (1997, January 7\u201310). Introduction to Modeling and Simulation. Proceedings of the 29th Conference on Winter Simulation, Atlanta, GA, USA.","DOI":"10.1145\/268437.268440"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2327","DOI":"10.1002\/spe.3277","article-title":"Faas-Sim: A Trace-Driven Simulation Framework for Serverless Edge Computing Platforms","volume":"53","author":"Raith","year":"2023","journal-title":"Soft. Pract. Exp."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Mampage, A., and Buyya, R. (2023, January 17\u201321). CloudSimSC: A Toolkit for Modeling and Simulation of Serverless Computing Environments. Proceedings of the 2023 IEEE International Conference on High Performance Computing & Communications, Data Science & Systems, Smart City & Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC\/DSS\/SmartCity\/DependSys), Melbourne, Australia.","DOI":"10.1109\/HPCC-DSS-SmartCity-DependSys60770.2023.00081"},{"key":"ref_33","unstructured":"Hasse, V.F. (2025, March 26). Emulation of Heterogeneous Kubernetes Clusters Using QEMU. Available online: https:\/\/data.goettingen-research-online.de\/dataset.xhtml?persistentId=doi:10.25625\/AQETJV."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"T09005","DOI":"10.1088\/1748-0221\/15\/09\/T09005","article-title":"Achieving Reliable UDP Transmission at 10 Gb\/s Using BSD Socket for Data Acquisition Systems","volume":"15","author":"Christensen","year":"2020","journal-title":"J. Instrum."},{"key":"ref_35","unstructured":"(2024, September 18). GitHub\u2014InfraBuilder\/benchmark-k8s-cni-2020-08: Results for 2020-08 using Flannel on Ubuntu 18.04. Available online: https:\/\/github.com\/InfraBuilder\/benchmark-k8s-cni-2020-08\/blob\/master\/results\/doc-flannel.u18.04-default\/doc-flannel.u18.04-default-run1.knbdata."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/6\/324\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:42:47Z","timestamp":1760031767000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/6\/324"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,29]]},"references-count":35,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["a18060324"],"URL":"https:\/\/doi.org\/10.3390\/a18060324","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2025,5,29]]}}}