{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,1]],"date-time":"2026-01-01T10:12:34Z","timestamp":1767262354107,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2020,8,8]],"date-time":"2020-08-08T00:00:00Z","timestamp":1596844800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>This paper presents the architecture, implementation and evaluation of a middleware support layer for NoSQL storage systems. Our middleware automatically selects performance and scalability tactics in terms of application specific workloads. Enterprises are turning to NoSQL storage technologies for their data-intensive computing and analytics applications. Comprehensive benchmarks of different Big Data platforms can help drive decisions which solutions to adopt. However, selecting the best performing technology, configuring the deployment for scalability and tuning parameters at runtime for an optimal service delivery remain challenging tasks, especially when application workloads evolve over time. Our middleware solves this problem at runtime by monitoring the data growth, changes in the read-write-query mix at run-time, as well as other system metrics that are indicative of sub-optimal performance. Our middleware employs supervised machine learning on historic and current monitoring information and corresponding configurations to select the best combinations of high-level tactics and adapt NoSQL systems to evolving workloads. This work has been driven by two real world case studies with different QoS requirements. The evaluation demonstrates that our middleware can adapt to unseen workloads of data-intensive applications, and automate the configuration of different families of NoSQL systems at runtime to optimize the performance and scalability of such applications.<\/jats:p>","DOI":"10.3390\/informatics7030029","type":"journal-article","created":{"date-parts":[[2020,8,10]],"date-time":"2020-08-10T05:07:23Z","timestamp":1597036043000},"page":"29","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Automated Configuration of NoSQL Performance and Scalability Tactics for Data-Intensive Applications"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6279-4430","authenticated-orcid":false,"given":"Davy","family":"Preuveneers","sequence":"first","affiliation":[{"name":"imec\u2014DistriNet, KU Leuven, Celestijnenlaan 200A, B-3001 Heverlee, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wouter","family":"Joosen","sequence":"additional","affiliation":[{"name":"imec\u2014DistriNet, KU Leuven, Celestijnenlaan 200A, B-3001 Heverlee, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,8,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Klein, J., Gorton, I., Ernst, N., Donohoe, P., Pham, K., and Matser, C. (2015, January 1). Performance Evaluation of NoSQL Databases: A Case Study. Proceedings of the 1st Workshop on Performance Analysis of Big Data Systems, Austin, TX, USA.","DOI":"10.1145\/2694730.2694731"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"112973","DOI":"10.1016\/j.eswa.2019.112973","article-title":"Feedback control loop design for workload change detection in self-tuning NoSQL wide column stores","volume":"142","author":"Mozaffari","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hillenbrand, A., St\u00f6rl, U., Levchenko, M., Nabiyev, S., and Klettke, M. (2020, January 20\u201324). Towards Self-Adapting Data Migration in the Context of Schema Evolution in NoSQL Databases. Proceedings of the 2020 IEEE 36th International Conference on Data Engineering Workshops (ICDEW), Dallas, TX, USA.","DOI":"10.1109\/ICDEW49219.2020.00013"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., and Sears, R. (2010, January 10\u201311). Benchmarking Cloud Serving Systems with YCSB. Proceedings of the 1st ACM Symposium on Cloud Computing, Indianapolis, IN, USA.","DOI":"10.1145\/1807128.1807152"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/MPRV.2008.80","article-title":"OpenStreetMap: User-Generated Street Maps","volume":"7","author":"Haklay","year":"2008","journal-title":"IEEE Pervasive Comput."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Fox, A., Eichelberger, C., Hughes, J., and Lyon, S. (2013, January 6\u20139). Spatio-temporal indexing in non-relational distributed databases. Proceedings of the 2013 IEEE International Conference on Big Data, Santa Clara, CA, USA.","DOI":"10.1109\/BigData.2013.6691586"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., and Vogels, W. (2007, January 14\u201317). Dynamo: Amazon\u2019s Highly Available Key-value Store. Proceedings of the Twenty-first ACM SIGOPS Symposium on Operating Systems Principles, Stevenson, WA, USA.","DOI":"10.1145\/1294261.1294281"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1016\/j.jss.2015.12.024","article-title":"Systematic scalability assessment for feature oriented multi-tenant services","volume":"116","author":"Preuveneers","year":"2016","journal-title":"J. Syst. Softw."},{"key":"ref_9","unstructured":"Bifet, A., and Gavald\u00e0, R. (September, January 31). Adaptive Learning from Evolving Data Streams. Proceedings of the 8th International Symposium on Intelligent Data Analysis: Advances in Intelligent Data Analysis VIII, Lyon, France."},{"key":"ref_10","first-page":"1601","article-title":"MOA: Massive Online Analysis","volume":"11","author":"Bifet","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hulten, G., Spencer, L., and Domingos, P. (2001, January 26\u201329). Mining time-changing data streams. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/502512.502529"},{"key":"ref_12","unstructured":"Marschall, M. (2013). Chef Infrastructure Automation Cookbook, Packt Publishing."},{"key":"ref_13","unstructured":"Gunther, N.J. (2007). Guerrilla Capacity Planning\u2014A Tactical Approach to Planning for Highly Scalable Applications and Services, Springer."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/j.knosys.2015.05.010","article-title":"MAG: A performance evaluation framework for database systems","volume":"85","author":"Khattab","year":"2015","journal-title":"Knowl.-Based Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Gencer, A.E., Bindel, D., Sirer, E.G., and van Renesse, R. (2015, January 7\u201311). Configuring Distributed Computations Using Response Surfaces. Proceedings of the 16th Annual Middleware Conference, Vancouver, BC, Canada.","DOI":"10.1145\/2814576.2814730"},{"key":"ref_16","unstructured":"Herodotou, H., Lim, H., Luo, G., Borisov, N., Dong, L., Cetin, F.B., and Babu, S. (2020, August 03). Starfish: A Self-tuning System for Big Data Analytics. CIDR. Available online: https:\/\/www-db.cs.wisc.edu\/cidr\/cidr2011\/Papers\/CIDR11_Paper36.pdf."},{"key":"ref_17","unstructured":"Koehler, J., Koehler, J., Giblin, C., Giblin, C., Gantenbein, D., Gantenbein, D., Hauser, R., and Hauser, R. (2003). On Autonomic Computing Architectures. Research Report (Computer Science) RZ 3487(#99302), IBM Research."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1380584.1380585","article-title":"A survey of autonomic computing-degrees, models, and applications","volume":"40","author":"Huebscher","year":"2008","journal-title":"ACM Comput. Surv."},{"key":"ref_19","unstructured":"Tesauro, G., and Kephart, J.O. (2004, January 17\u201318). Utility Functions in Autonomic Systems. Proceedings of the First International Conference on Autonomic Computing, Washington, DC, USA."},{"key":"ref_20","first-page":"115","article-title":"Achieving self-managed deployment in a distributed environment","volume":"11","author":"Deb","year":"2011","journal-title":"J. Comp. Methods Sci. Eng."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Hu, Y., Wong, J., Iszlai, G., and Litoiu, M. (2009, January 2\u20135). Resource provisioning for cloud computing. Proceedings of the 2009 Conference of the Center for Advanced Studies on Collaborative Research, Toronto, ON, Canada.","DOI":"10.1145\/1723028.1723041"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Koehler, M., and Benkner, S. (2012, January 5\u20138). Design of an Adaptive Framework for Utility-Based Optimization of Scientific Applications in the Cloud. Proceedings of the 2012 IEEE\/ACM Fifth International Conference on Utility and Cloud Computing, Chicago, IL, USA.","DOI":"10.1109\/UCC.2012.48"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.jpdc.2015.03.001","article-title":"Incremental Dataflow Execution, Resource Efficiency and Probabilistic Guarantees with Fuzzy Boolean Nets","volume":"79","author":"Esteves","year":"2015","journal-title":"J. Parallel Distrib. Comput."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Van Aken, D., Pavlo, A., Gordon, G.J., and Zhang, B. (2017, January 14\u201319). Automatic Database Management System Tuning Through Large-scale Machine Learning. Proceedings of the 2017 ACM International Conference on Management of Data, Chicago, IL, USA.","DOI":"10.1145\/3035918.3064029"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Cruz, F., Maia, F., Matos, M., Oliveira, R., Paulo, J., Pereira, J., and Vila\u00e7a, R. (2013, January 15). Met: Workload aware elasticity for nosql. Proceedings of the 8th ACM European Conference on Computer Systems, Prague, Czech Republic.","DOI":"10.1145\/2465351.2465370"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"48","DOI":"10.14778\/1920841.1920853","article-title":"Schism: A Workload-driven Approach to Database Replication and Partitioning","volume":"3","author":"Curino","year":"2010","journal-title":"Proc. VLDB Endow."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Couceiro, M., Chandrasekara, G., Bravo, M., Hiltunen, M., Romano, P., and Rodrigues, L. (2015, January 7\u201311). Q-OPT: Self-tuning Quorum System for Strongly Consistent Software Defined Storage. Proceedings of the 16th Annual Middleware Conference, Vancouver, BC, Canada.","DOI":"10.1145\/2814576.2814809"},{"key":"ref_28","unstructured":"McKenzie, M., Fan, H., and Golab, W. (2015). Continuous Partial Quorums for Consistency-Latency Tuning in Distributed NoSQL Storage Systems. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Voigt, H., Kissinger, T., and Lehner, W. (2013, January 29\u201331). Smix: Self-managing indexes for dynamic workloads. Proceedings of the 25th International Conference on Scientific and Statistical Database Management, Baltimore, MD, USA.","DOI":"10.1145\/2484838.2484862"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Hutter, F., Kotthoff, L., and Vanschoren, J. (2019). Auto-WEKA: Automatic Model Selection and Hyperparameter Optimization in WEKA. Automated Machine Learning: Methods, Systems, Challenges, Springer International Publishing.","DOI":"10.1007\/978-3-030-05318-5_4"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Hutter, F., Kotthoff, L., and Vanschoren, J. (2019). Auto-sklearn: Efficient and Robust Automated Machine Learning. Automated Machine Learning: Methods, Systems, Challenges, Springer International Publishing.","DOI":"10.1007\/978-3-030-05318-5"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1109\/MC.2012.37","article-title":"CAP Twelve Years Later: How the \u201cRules\u201d Have Changed","volume":"45","author":"Brewer","year":"2012","journal-title":"Computer"},{"key":"ref_33","unstructured":"Jamshidi, P., Velez, M., K\u00e4stner, C., Siegmund, N., and Kawthekar, P. Transfer Learning for Improving Model Predictions in Highly Configurable Software. Proceedings of the 12th International Symposium on Software Engineering for Adaptive and Self-Managing Systems."}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/7\/3\/29\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:58:15Z","timestamp":1760176695000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/7\/3\/29"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,8]]},"references-count":33,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["informatics7030029"],"URL":"https:\/\/doi.org\/10.3390\/informatics7030029","relation":{},"ISSN":["2227-9709"],"issn-type":[{"type":"electronic","value":"2227-9709"}],"subject":[],"published":{"date-parts":[[2020,8,8]]}}}