{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,23]],"date-time":"2026-06-23T06:45:58Z","timestamp":1782197158119,"version":"3.54.5"},"reference-count":32,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2019,2,14]],"date-time":"2019-02-14T00:00:00Z","timestamp":1550102400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No. 61602525 and 61572525"],"award-info":[{"award-number":["No. 61602525 and 61572525"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>A distributed data stream processing system handles real-time, changeable and sudden streaming data load. Its elastic resource allocation has become a fundamental and challenging problem with a fixed strategy that will result in waste of resources or a reduction in QoS (quality of service). Spark Streaming as an emerging system has been developed to process real time stream data analytics by using micro-batch approach. In this paper, first, we propose an improved SVR (support vector regression) based stream data load prediction scheme. Then, we design a spark-based maximum sustainable throughput of time window (MSTW) performance model to find the optimized number of virtual machines. Finally, we present a resource scaling algorithm TWRES (time window resource elasticity scaling algorithm) with MSTW constraint and streaming data load prediction. The evaluation results show that TWRES could improve resource utilization and mitigate SLA (service level agreement) violation.<\/jats:p>","DOI":"10.3390\/a12020037","type":"journal-article","created":{"date-parts":[[2019,2,14]],"date-time":"2019-02-14T11:54:13Z","timestamp":1550145253000},"page":"37","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Stream Data Load Prediction for Resource Scaling Using Online Support Vector Regression"],"prefix":"10.3390","volume":"12","author":[{"given":"Zhigang","family":"Hu","sequence":"first","affiliation":[{"name":"School of Software, Central South University, Changsha 410075, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1744-8763","authenticated-orcid":false,"given":"Hui","family":"Kang","sequence":"additional","affiliation":[{"name":"School of Software, Central South University, Changsha 410075, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Meiguang","family":"Zheng","sequence":"additional","affiliation":[{"name":"School of Software, Central South University, Changsha 410075, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,2,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1007\/s10723-014-9314-7","article-title":"A Review of Auto-scaling Techniques for Elastic Applications in Cloud Environments","volume":"12","author":"Lozano","year":"2014","journal-title":"J. Grid Comput."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Imai, S., Patterson, S., and Varela, C.A. (2017, January 14\u201317). Maximum Sustainable Throughput Prediction for Data Stream Processing over Public Clouds. Proceedings of the 2017 17th IEEE\/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), Madrid, Spain.","DOI":"10.1109\/CCGRID.2017.105"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Fang, W., Lu, Z., Wu, J., and Cao, Z. (2012, January 24\u201329). RPPS: A Novel Resource Prediction and Provisioning Scheme in Cloud Data Center. Proceedings of the IEEE Ninth International Conference on Services Computing, Honolulu, HI, USA.","DOI":"10.1109\/SCC.2012.47"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.future.2011.05.027","article-title":"Empirical prediction models for adaptive resource provisioning in the cloud","volume":"28","author":"Islam","year":"2012","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1109\/TSC.2011.61","article-title":"Resource provisioning with budget constraints for adaptive applications in cloud environments","volume":"5","author":"Zhu","year":"2012","journal-title":"IEEE Trans. Serv. Comput."},{"key":"ref_6","unstructured":"Teng, L., Jian, T., and Jielong, X. (November, January 29). A predictive scheduling framework for fast and distributed stream data processing. Proceedings of the 2015 IEEE International Conference on Big Data, Santa Clara, CA, USA."},{"key":"ref_7","unstructured":"Tizianod, M., and Gabriele, M. (2017, January 6\u20138). Elastic Scaling for Distributed Latency-Sensitive Data Stream Operators. Proceedings of the 2017 Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), St. Petersburg, Russia."},{"key":"ref_8","unstructured":"Bj\u00f6rn, L., Peter, J., and Odej, K. (July, January 29). Elastic Stream Processing with Latency Guarantees. Proceedings of the 2015 IEEE 35th International Conference on Distributed Computing Systems, Columbus, OH, USA."},{"key":"ref_9","unstructured":"Raul, C.F., Matteo, M., Evangelia, K., and Peter, P. (2013, January 22\u201327). Integrating scale out and fault tolerance in stream processing using operator state management. Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, New York, NY, USA."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Liu, N., Li, Z., Xu, J., Xu, Z., Lin, S., Qiu, Q., Tang, J., and Wang, Y. (2017, January 5\u20138). A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning. Proceedings of the 37th IEEE International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA.","DOI":"10.1109\/ICDCS.2017.123"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Fu, T.Z., Ding, J., Ma, R.T., Winslett, M., Yang, Y., and Zhang, Z. (July, January 29). DRS: Dynamic Resource Scheduling for Real-Time Analytics over Fast Streams. Proceedings of the 2015 IEEE International Conference on Distributed Computing Systems, Columbus, OH, USA.","DOI":"10.1109\/ICDCS.2015.49"},{"key":"ref_12","unstructured":"Ruben, M., Boris, K., and Kurt, R. (2014, January 27\u201330). Meeting predictable buffer limits in the parallel execution of event processing operators. Proceedings of the 2014 IEEE International Conference on Big Data, Washington, DC, USA."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1109\/TCC.2015.2394316","article-title":"Reactive Resource Provisioning Heuristics for Dynamic Data flows on Cloud Infrastructure","volume":"3","author":"Kumbhare","year":"2015","journal-title":"IEEE Trans. Cloud Comput."},{"key":"ref_14","unstructured":"Alok, G.K., Yogesh, S., and Viktor, K.P. (2014, January 26\u201329). PLASTICC: Predictive Look-Ahead Scheduling for Continuous Data flows on Clouds. Proceedings of the 2014 IEEE\/ACM International Symposium on Cluster, Cloud and Grid Computing, Chicago, IL, USA."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"878","DOI":"10.1109\/TPDS.2014.2313335","article-title":"Profiling-based workload consolidation & migration in VDCs","volume":"26","author":"Ye","year":"2015","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_16","first-page":"273","article-title":"Towards fair and efficient SMP VM scheduling","volume":"49","author":"Rao","year":"2014","journal-title":"Assoc. Comput. Mach."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Madsen, K.G.S., and Zhou, Y. (2015, January 24\u201328). Dynamic resource management in a massively parallel stream processing engine. Proceedings of the 24th ACM International Conference on Information and Knowledge Management, Indianapolis, IN, USA.","DOI":"10.1145\/2806416.2806449"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wu, Y., and Tan, K.L. (2015, January 13\u201317). Chrono stream: Elastic stateful stream computation in the cloud. Proceedings of the 2015 IEEE 31st International Conference on Data Engineering, Seoul, Korea.","DOI":"10.1109\/ICDE.2015.7113328"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2187","DOI":"10.1002\/cpe.3495","article-title":"Adaptive model predictive control of autonomic distributed parallel computations with variable horizons and switching costs","volume":"28","author":"Mencagli","year":"2016","journal-title":"Concurrency Comput. Pract. Exp."},{"key":"ref_20","unstructured":"Jianfei, R., Qinghua, Z., and Bo, D. (2015, January 7\u201311). Optimal resource provisioning approach based on cost modeling for spark applications in public clouds. Proceedings of the Doctoral Symposium of the 16th International Middleware Conference, Vancouver, BC, Canada."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Park, J., Lee, D., Kim, B., Huh, J., and Maeng, S. (2012, January 18\u201322). Locality-aware dynamic vm reconfiguration on mapreduce clouds. Proceedings of the 21st International Symposium on High-Performance Parallel and Distributed Computing (HPDC), Delft, The Netherlands.","DOI":"10.1145\/2287076.2287082"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1016\/j.ijpe.2017.08.012","article-title":"A multi-objective optimization model for the design of an effective decarbonized supply chain in mining","volume":"193","year":"2017","journal-title":"Int. J. Prod. Econ."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"65635","DOI":"10.1109\/ACCESS.2018.2874439","article-title":"A Comprehensive Evaluation of Weak and Strong Mutation Mechanisms in Evolutionary Algorithms for Truck Scheduling at Cross-Docking Terminals","volume":"6","author":"Dulebenets","year":"2018","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"748","DOI":"10.1016\/j.ijpe.2017.09.002","article-title":"Optimization in inventory-routing problem with planned transshipment: A case study in the retail industry","volume":"193","author":"Peres","year":"2017","journal-title":"Int. J. Prod. Econ."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1016\/j.engappai.2017.11.009","article-title":"A collaborative agreement for berth allocation under excessive demand","volume":"69","author":"Dulebenets","year":"2018","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.ejor.2018.11.033","article-title":"A hybrid Lagrangian metaheuristic for the cross-docking flow shop scheduling problem","volume":"275","author":"Fonseca","year":"2018","journal-title":"Eur. J. Oper. Res."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"653","DOI":"10.1016\/j.ejor.2017.06.056","article-title":"A hybrid of adaptive large neighborhood search and tabu search for the order-batching problem","volume":"264","author":"Kramer","year":"2018","journal-title":"Eur. J. Oper. Res."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1016\/j.ijpe.2017.10.027","article-title":"A comprehensive multi-objective optimization model for the vessel scheduling problem in liner shipping","volume":"196","author":"Dulebenets","year":"2018","journal-title":"Int. J. Prod. Econ."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/65.844498","article-title":"A workload characterization study of the 1998 World Cup Web site","volume":"14","author":"Arlitt","year":"2000","journal-title":"Netw. IEEE"},{"key":"ref_30","unstructured":"(2017, February 20). Poland Electric Demand time series, Darwin Sea Level Pressures, and Darwin Sea Are Tested. Available online: http:\/\/www.cis.ht.fi\/projects\/tsp\/?page=Timeseries."},{"key":"ref_31","unstructured":"Intel Corporation (2017, February 20). Storm benchmark. Available online: https:\/\/github.com\/intel-hadoop\/storm-benchmark."},{"key":"ref_32","unstructured":"(2017, February 26). The Project Gutenberg Ebook of the Adventure of Tom Sawyer. Available online: https:\/\/www.gutenberg.org\/files\/74\/74-h\/74-h.htm."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/12\/2\/37\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:32:05Z","timestamp":1760185925000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/12\/2\/37"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,2,14]]},"references-count":32,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2019,2]]}},"alternative-id":["a12020037"],"URL":"https:\/\/doi.org\/10.3390\/a12020037","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,2,14]]}}}