{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,20]],"date-time":"2026-09-20T01:24:06Z","timestamp":1789867446918,"version":"4.0.1"},"reference-count":51,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2018,9,7]],"date-time":"2018-09-07T00:00:00Z","timestamp":1536278400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>The capability of efficiently processing the data streams emitted by nowadays ubiquitous sensing devices enables the development of new intelligent services. Data Stream Processing (DSP) applications allow for processing huge volumes of data in near real-time. To keep up with the high volume and velocity of data, these applications can elastically scale their execution on multiple computing resources to process the incoming data flow in parallel. Being that data sources and consumers are usually located at the network edges, nowadays the presence of geo-distributed computing resources represents an attractive environment for DSP. However, controlling the applications and the processing infrastructure in such wide-area environments represents a significant challenge. In this paper, we present a hierarchical solution for the autonomous control of elastic DSP applications and infrastructures. It consists of a two-layered hierarchical solution, where centralized components coordinate subordinated distributed managers, which, in turn, locally control the elastic adaptation of the application components and deployment regions. Exploiting this framework, we design several self-adaptation policies, including reinforcement learning based solutions. We show the benefits of the presented self-adaptation policies with respect to static provisioning solutions, and discuss the strengths of reinforcement learning based approaches, which learn from experience how to optimize the application performance and resource allocation.<\/jats:p>","DOI":"10.3390\/a11090134","type":"journal-article","created":{"date-parts":[[2018,9,7]],"date-time":"2018-09-07T11:47:41Z","timestamp":1536320861000},"page":"134","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Multi-Level Elasticity for Wide-Area Data Streaming Systems: A Reinforcement Learning Approach"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8233-4570","authenticated-orcid":false,"given":"Gabriele","family":"Russo\u00a0Russo","sequence":"first","affiliation":[{"name":"Department of Civil Engineering and Computer Science Engineering, University of Rome Tor Vergata, 00133 Rome, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9519-9387","authenticated-orcid":false,"given":"Matteo","family":"Nardelli","sequence":"additional","affiliation":[{"name":"Department of Civil Engineering and Computer Science Engineering, University of Rome Tor Vergata, 00133 Rome, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6870-7083","authenticated-orcid":false,"given":"Valeria","family":"Cardellini","sequence":"additional","affiliation":[{"name":"Department of Civil Engineering and Computer Science Engineering, University of Rome Tor Vergata, 00133 Rome, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7461-6276","authenticated-orcid":false,"given":"Francesco","family":"Lo\u00a0Presti","sequence":"additional","affiliation":[{"name":"Department of Civil Engineering and Computer Science Engineering, University of Rome Tor Vergata, 00133 Rome, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,9,7]]},"reference":[{"key":"ref_1","unstructured":"Liu, X., and Buyya, R. (2017). Performance-Oriented Deployment of Streaming Applications on Cloud. IEEE Trans. Big Data."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Jerzak, Z., and Ziekow, H. (July, January 29). The DEBS 2015 Grand Challenge. Proceedings of the 9th ACM International Conference on Distributed Event-Based Systems (DEBS), Oslo, Norway.","DOI":"10.1145\/2675743.2772598"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jnca.2017.12.001","article-title":"Distributed Data Stream Processing and Edge Computing: A Survey on Resource Elasticity and Future Directions","volume":"103","author":"Veith","year":"2018","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1145\/2528412","article-title":"A Catalog of Stream Processing Optimizations","volume":"46","author":"Hirzel","year":"2014","journal-title":"ACM Comput. Surv."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"572","DOI":"10.1109\/TPDS.2017.2762683","article-title":"Elastic Symbiotic Scaling of Operators and Resources in Stream Processing Systems","volume":"29","author":"Lombardi","year":"2018","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_6","first-page":"24","article-title":"A Stepwise Auto-Profiling Method for Performance Optimization of Streaming Applications","volume":"12","author":"Liu","year":"2018","journal-title":"ACM Trans. Auton. Adapt. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"e4334","DOI":"10.1002\/cpe.4334","article-title":"Optimal Operator Deployment and Replication for Elastic Distributed Data Stream Processing","volume":"30","author":"Cardellini","year":"2018","journal-title":"Concurr. Comput. Pract. Exp."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1016\/j.future.2018.05.025","article-title":"Decentralized Self-Adaptation for Elastic Data Stream Processing","volume":"87","author":"Cardellini","year":"2018","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_9","unstructured":"Hochreiner, C., V\u00f6gler, M., Schulte, S., and Dustdar, S. (July, January 27). Elastic Stream Processing for the Internet of Things. Proceedings of the IEEE 9th International Conference on Cloud Computing (CLOUD), San Francisco, CA, USA."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Pietzuch, P., Ledlie, J., Shneidman, J., Roussopoulos, M., Welsh, M., and Seltzer, M. (2006, January 3\u20137). Network-Aware Operator Placement for Stream-Processing Systems. Proceedings of the 22nd International Conference on Data Engineering (ICDE), Atlanta, GA, USA.","DOI":"10.1109\/ICDE.2006.105"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Rizou, S., Durr, F., and Rothermel, K. (2010, January 2\u20135). Solving the Multi-Operator Placement Problem in Large-Scale Operator Networks. Proceedings of the 19th International Conference on Computer Communications and Networks (ICCCN), Zurich, Switzerland.","DOI":"10.1109\/ICCCN.2010.5560127"},{"key":"ref_12","unstructured":"Cardellini, V., Grassi, V., Lo Presti, F., and Nardelli, M. (July, January 29). Distributed QoS-Aware Scheduling in Storm. Proceedings of the 9th ACM International Conference on Distributed Event-Based Systems (DEBS), Oslo, Norway."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Nardelli, M., Russo Russo, G., Cardellini, V., and Lo Presti, F. (2018). A Multi-Level Elasticity Framework for Distributed Data Stream Processing. Euro-Par 2018: Parallel Processing Workshops, Springer.","DOI":"10.1007\/978-3-030-10549-5_5"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An Introduction, MIT Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1447","DOI":"10.1109\/TPDS.2013.295","article-title":"Elastic Scaling for Data Stream Processing","volume":"25","author":"Gedik","year":"2014","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1109\/MC.2003.1160055","article-title":"The Vision of Autonomic Computing","volume":"36","author":"Kephart","year":"2003","journal-title":"IEEE Comput."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2351","DOI":"10.1109\/TPDS.2012.24","article-title":"StreamCloud: An Elastic and Scalable Data Streaming System","volume":"23","author":"Gulisano","year":"2012","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_18","unstructured":"Aniello, L., Baldoni, R., and Querzoni, L. (July, January 29). Adaptive Online Scheduling in Storm. Proceedings of the 7th ACM International Conference on Distributed Event-Based Systems (DEBS), Arlington, Texas, USA."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3338","DOI":"10.1109\/TNET.2017.2741969","article-title":"DRS: Auto-Scaling for Real-Time Stream Analytics","volume":"25","author":"Fu","year":"2017","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Madsen, K.G.S., Zhou, Y., and Cao, J. (2017, January 19\u201322). Integrative Dynamic Reconfiguration in a Parallel Stream Processing Engine. Proceedings of the IEEE 33rd International Conference on Data Engineering (ICDE), San Diego, CA, USA.","DOI":"10.1109\/ICDE.2017.81"},{"key":"ref_21","unstructured":"Xu, J., Chen, Z., Tang, J., and Su, S. (July, January 30). T-Storm: Traffic-Aware Online Scheduling in Storm. Proceedings of the IEEE 34th International Conference on Distributed Computing Systems (ICDCS), Madrid, Spain."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"13:1","DOI":"10.1145\/2903146","article-title":"A Game-Theoretic Approach for Elastic Distributed Data Stream Processing","volume":"11","author":"Mencagli","year":"2016","journal-title":"ACM Trans. Auton. Adapt. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Ooi, B.C., Tan, K.L., and Wu, J. (November, January 29). Efficient Dynamic Operator Placement in a Locally Distributed Continuous Query System. Proceedings of the 2006 Confederated International Conference on the Move to Meaningful Internet Systems: CoopIS, DOA, GADA, and ODBASE, Montpellier, France.","DOI":"10.1007\/11914853_5"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Aberer, K., and Tan, K.L. (2008, January 1\u20135). Toward Massive Query Optimization in Large-Scale Distributed Stream Systems. Proceedings of the ACM\/IFIP\/USENIX International Conference on Distributed Systems Platforms and Open Distributed Processing (Middleware), Leuven, Belgium.","DOI":"10.1007\/978-3-540-89856-6_17"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Papageorgiou, A., Poormohammady, E., and Cheng, B. (July, January 27). Edge-Computing-Aware Deployment of Stream Processing Tasks Based on Topology-External Information: Model, Algorithms, and a Storm-Based Prototype. Proceedings of the 2016 IEEE International Congress on Big Data (BigData Congress), San Francisco, CA, USA.","DOI":"10.1109\/BigDataCongress.2016.40"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Sajjad, H.P., Danniswara, K., Al-Shishtawy, A., and Vlassov, V. (2016, January 27\u201328). SpanEdge: Towards Unifying Stream Processing over Central and Near-the-Edge Data Centers. Proceedings of the 2016 IEEE\/ACM Symposium on Edge Computing (SEC), Washington, DC, USA.","DOI":"10.1109\/SEC.2016.17"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Saurez, E., Hong, K., Lillethun, D., Ramachandran, U., and Ottenw\u00e4lder, B. (2016, January 20\u201324). Incremental Deployment and Migration of Geo-distributed Situation Awareness Applications in the Fog. Proceedings of the 10th ACM International Conference on Distributed and Event-Based Systems (DEBS), Irvine, CA, USA.","DOI":"10.1145\/2933267.2933317"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1109\/MPRV.2015.32","article-title":"Edge Analytics in the Internet of Things","volume":"14","author":"Satyanarayanan","year":"2015","journal-title":"IEEE Pervasive Comput."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1109\/MIC.2017.2911430","article-title":"A Serverless Real-Time Data Analytics Platform for Edge Computing","volume":"21","author":"Nastic","year":"2017","journal-title":"IEEE Internet Comput."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2004","DOI":"10.1109\/TPDS.2018.2812177","article-title":"Firework: Data Processing and Sharing for Hybrid Cloud-Edge Analytics","volume":"29","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_31","unstructured":"To, Q., Soto, J., and Markl, V. (arXiv, 2017). A Survey of State Management in Big Data Processing Systems, arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Cardellini, V., Nardelli, M., and Luzi, D. (2016, January 18\u201322). Elastic Stateful Stream Processing in Storm. Proceedings of the 2016 International Conference on High Performance Computing & Simulation (HPCS), Innsbruck, Austria.","DOI":"10.1109\/HPCSim.2016.7568388"},{"key":"ref_33","unstructured":"Fernandez, R.C., Migliavacca, M., Kalyvianaki, E., and Pietzuch, P. (2013, January 22\u201327). Integrating Scale out and Fault Tolerance in Stream Processing Using Operator State Management. Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, New York, NY, USA."},{"key":"ref_34","unstructured":"Heinze, T., Pappalardo, V., Jerzak, Z., and Fetzer, C. (April, January 31). Auto-Scaling Techniques for Elastic Data Stream Processing. Proceedings of the IEEE 30th International Conference on Data Engineering Workshops, Chicago, IL, USA."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"De Matteis, T., and Mencagli, G. (2017, January 6\u20138). Elastic Scaling for Distributed Latency-sensitive Data Stream Operators. Proceedings of the 25th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), St. Petersburg, Russia.","DOI":"10.1109\/PDP.2017.31"},{"key":"ref_36","unstructured":"Lohrmann, B., Janacik, P., and Kao, O. (July, January 29). Elastic Stream Processing with Latency Guarantees. Proceedings of the IEEE 35th International Conference on Distributed Computing Systems (ICDCS), Columbus, OH, USA."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1016\/j.future.2017.09.004","article-title":"Elastic-PPQ: A Two-Level Autonomic System for Spatial Preference Query Processing over Dynamic Data Streams","volume":"79","author":"Mencagli","year":"2018","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Xu, L., Peng, B., and Gupta, I. (2016, January 4\u20138). Stela: Enabling Stream Processing Systems to Scale-in and Scale-out On-demand. Proceedings of the 2016 IEEE International Conference on Cloud Engineering (IC2E), Berlin, Germany.","DOI":"10.1109\/IC2E.2016.38"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Heinze, T., Roediger, L., Meister, A., Ji, Y., Jerzak, Z., and Fetzer, C. (2015, January 27\u201329). Online Parameter Optimization for Elastic Data Stream Processing. Proceedings of the Sixth ACM Symposium on Cloud Computing (SoCC), Kohala Coast, HI, USA.","DOI":"10.1145\/2806777.2806847"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Kotto Kombi, R., Lumineau, N., and Lamarre, P. (2017, January 5\u20138). A Preventive Auto-Parallelization Approach for Elastic Stream Processing. Proceedings of the IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA.","DOI":"10.1109\/ICDCS.2017.253"},{"key":"ref_41","first-page":"97","article-title":"Auto-Scaling in Data Stream Processing Applications: A Model Based Reinforcement Learning Approach","volume":"Volume 825","author":"Cardellini","year":"2017","journal-title":"New Frontiers in Quantitative Methods in Informatics. InfQ 2017. Communications in Computer and Information Science"},{"key":"ref_42","first-page":"61","article-title":"A Survey and Taxonomy of Self-Aware and Self-Adaptive Cloud Autoscaling Systems","volume":"51","author":"Chen","year":"2018","journal-title":"ACM Comput. Surv."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1007\/s10723-014-9314-7","article-title":"A Review of Auto-scaling Techniques for Elastic Applications in Cloud Environments","volume":"12","author":"Lozano","year":"2014","journal-title":"J. Grid Comput."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1109\/MCC.2016.66","article-title":"Managing Uncertainty in Autonomic Cloud Elasticity Controllers","volume":"3","author":"Jamshidi","year":"2016","journal-title":"IEEE Cloud Comput."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1656","DOI":"10.1002\/cpe.2864","article-title":"Applying Reinforcement Learning Towards Automating Resource Allocation and Application Scalability in the Cloud","volume":"25","author":"Barrett","year":"2012","journal-title":"Concurr. Comput. Pract. Exp."},{"key":"ref_46","unstructured":"Dutreilh, X., Kirgizov, S., Melekhova, O., Malenfant, J., Rivierre, N., and Truck, I. (2011, January 22\u201327). Using Reinforcement Learning for Autonomic Resource Allocation in Clouds: Towards a Fully Automated Workflow. Proceedings of the 7th International Conference on Autonomic and Autonomous Systems, Venice, Italy."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1007\/s10586-007-0035-6","article-title":"On the use of hybrid reinforcement learning for autonomic resource allocation","volume":"10","author":"Tesauro","year":"2007","journal-title":"Cluster Comput."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Arabnejad, H., Pahl, C., Jamshidi, P., and Estrada, G. (2017, January 14\u201317). A Comparison of Reinforcement Learning Techniques for Fuzzy Cloud Auto-Scaling. Proceedings of the 17th IEEE\/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), Madrid, Spain.","DOI":"10.1109\/CCGRID.2017.15"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1007\/978-3-642-35813-5_4","article-title":"On Patterns for Decentralized Control in Self-Adaptive Systems","volume":"Volume 7475","author":"Weyns","year":"2018","journal-title":"Lecture Notes in Computer Science"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Yoon, K.P., and Hwang, C.L. (1995). Multiple Attribute Decision Making: An Introduction, SAGE Press.","DOI":"10.4135\/9781412985161"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Bellman, R. (2010). Dynamic Programming, Princeton University Press.","DOI":"10.1515\/9781400835386"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/11\/9\/134\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:19:19Z","timestamp":1760195959000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/11\/9\/134"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,9,7]]},"references-count":51,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2018,9]]}},"alternative-id":["a11090134"],"URL":"https:\/\/doi.org\/10.3390\/a11090134","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,9,7]]}}}