{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:35:11Z","timestamp":1750221311799,"version":"3.41.0"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2018,4,10]],"date-time":"2018-04-10T00:00:00Z","timestamp":1523318400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Swedish Government\u2019s strategic research project eSSENCE"},{"name":"Dutch projects Vidi MagnaData and KIEM KIESA"},{"name":"Research Group of the Standard Performance Evaluation Corporation"},{"name":"Swedish Research Council (VR) project Cloud Control"},{"name":"Commit and the Commit projects IV-E and Commissioner"},{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["#1422245"],"award-info":[{"award-number":["#1422245"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"German Research Foundation","doi-asserted-by":"crossref","award":["KO 3445\/11-1"],"award-info":[{"award-number":["KO 3445\/11-1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Model. Perform. Eval. Comput. Syst."],"published-print":{"date-parts":[[2018,6,30]]},"abstract":"<jats:p>\n            Elasticity is one of the main features of cloud computing allowing customers to scale their resources based on the workload. Many autoscalers have been proposed in the past decade to decide on behalf of cloud customers when and how to provision resources to a cloud application based on the workload utilizing cloud elasticity features. However, in prior work, when a new policy is proposed, it is seldom compared to the state-of-the-art, and is often compared only to static provisioning using a predefined quality of service target. This reduces the ability of cloud customers and of cloud operators to choose and deploy an autoscaling policy, as there is seldom enough analysis on the performance of the autoscalers in different operating conditions and with different applications. In our work, we conduct an\n            <jats:italic>experimental<\/jats:italic>\n            performance evaluation of autoscaling policies, using as application model workflows, a popular formalism for automating resource management for applications with well-defined yet complex structures. We present a detailed comparative study of general state-of-the-art autoscaling policies, along with two new workflow-specific policies. To understand the performance differences between the seven policies, we conduct various experiments and compare their performance in both pairwise and group comparisons. We report both individual and aggregated metrics. As many workflows have deadline requirements on the tasks, we study the effect of autoscaling on workflow deadlines. Additionally, we look into the effect of autoscaling on the accounted and hourly based charged costs, and we evaluate performance variability caused by the autoscaler selection for each group of workflow sizes. Our results highlight the trade-offs between the suggested policies, how they can impact meeting the deadlines, and how they perform in different operating conditions, thus enabling a better understanding of the current state-of-the-art.\n          <\/jats:p>","DOI":"10.1145\/3164537","type":"journal-article","created":{"date-parts":[[2018,4,11]],"date-time":"2018-04-11T12:31:03Z","timestamp":1523449863000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["An Experimental Performance Evaluation of Autoscalers for Complex Workflows"],"prefix":"10.1145","volume":"3","author":[{"given":"Alexey","family":"Ilyushkin","sequence":"first","affiliation":[{"name":"Delft University of Technology, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ahmed","family":"Ali-Eldin","sequence":"additional","affiliation":[{"name":"Ume\u00e5 University and UMass, Amherst, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nikolas","family":"Herbst","sequence":"additional","affiliation":[{"name":"University of W\u00fcrzburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andr\u00e9","family":"Bauer","sequence":"additional","affiliation":[{"name":"University of W\u00fcrzburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1364-8127","authenticated-orcid":false,"given":"Alessandro V.","family":"Papadopoulos","sequence":"additional","affiliation":[{"name":"M\u00e4lardalen University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dick","family":"Epema","sequence":"additional","affiliation":[{"name":"Delft University of Technology, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexandru","family":"Iosup","sequence":"additional","affiliation":[{"name":"Vrije Universiteit Amsterdam and Delft University of Technology, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2018,4,10]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevD.77.062002"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2011.303"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/NOMS.2012.6211900"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2287036.2287044"},{"key":"e_1_2_1_5_1","unstructured":"Ahmed Ali-Eldin etal 2013. Workload Classification for Efficient Auto-Scaling of Cloud Resources. Technical Report. Ume\u00e5 University Lund University.  Ahmed Ali-Eldin et al. 2013. Workload Classification for Efficient Auto-Scaling of Cloud Resources. Technical Report. Ume\u00e5 University Lund University."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/502912.502915"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISPA.2012.94"},{"volume-title":"Proceedings of WORKS Workshop.","author":"Shishir","key":"e_1_2_1_8_1","unstructured":"Shishir Bharathi et al. 2008. Characterization of scientific workflows . In Proceedings of WORKS Workshop. Shishir Bharathi et al. 2008. Characterization of scientific workflows. In Proceedings of WORKS Workshop."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2011.05.001"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/502059.502045"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICEBE.2009.45"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2017.01.011"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2089002.2089003"},{"key":"e_1_2_1_14_1","doi-asserted-by":"crossref","unstructured":"Rafael Ferreira Da Silva etal 2015. Online task resource consumption prediction for scientific workflows. Parallel Process. Lett. 25 (2015).  Rafael Ferreira Da Silva et al. 2015. Online task resource consumption prediction for scientific workflows. Parallel Process. Lett. 25 (2015).","DOI":"10.1142\/S0129626415410030"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/74.2.432"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2016.05.011"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2644865.2541941"},{"key":"e_1_2_1_18_1","doi-asserted-by":"crossref","unstructured":"Qiu Dishan et al. 2013. A dynamic scheduling method of earth-observing satellites by employing rolling horizon strategy. Sci. World J. 2013 (2013).  Qiu Dishan et al. 2013. A dynamic scheduling method of earth-observing satellites by employing rolling horizon strategy. Sci. World J. 2013 (2013).","DOI":"10.1155\/2013\/304047"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CCGRID.2009.30"},{"volume-title":"Proceedings of IEEE Cluster.","author":"Lipu","key":"e_1_2_1_20_1","unstructured":"Lipu Fei et al. 2014. KOALA-C: A task allocator for integrated multicluster and multicloud environments . In Proceedings of IEEE Cluster. Lipu Fei et al. 2014. KOALA-C: A task allocator for integrated multicluster and multicloud environments. In Proceedings of IEEE Cluster."},{"volume-title":"Workload Modeling for Computer Systems Performance Evaluation","author":"Feitelson Dror G.","key":"e_1_2_1_21_1","unstructured":"Dror G. Feitelson . 2015. Workload Modeling for Computer Systems Performance Evaluation . Cambridge University Press . Dror G. Feitelson. 2015. Workload Modeling for Computer Systems Performance Evaluation. Cambridge University Press."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/IC2E.2014.25"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/5666.5673"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2382553.2382556"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICAC.2005.21"},{"key":"e_1_2_1_26_1","unstructured":"Nikolas Herbst et al. 2016. Ready for Rain? A View from SPEC Research on the Future of Cloud Metrics. Technical Report. SPEC Research Group Cloud Working Group.  Nikolas Herbst et al. 2016. Ready for Rain? A View from SPEC Research on the Future of Cloud Metrics. Technical Report. SPEC Research Group Cloud Working Group."},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of ICAC.","author":"Herbst Nikolas","year":"2013","unstructured":"Nikolas Herbst , Samuel Kounev , and Ralf Reussner . 2013 . Elasticity in cloud computing: What it is, and What it is Not . In Proceedings of ICAC. Nikolas Herbst, Samuel Kounev, and Ralf Reussner. 2013. Elasticity in cloud computing: What it is, and What it is Not. In Proceedings of ICAC."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3030207.3030214"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/CCGrid.2015.27"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2011.66"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2010.10.016"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2443416.2443420"},{"key":"e_1_2_1_33_1","first-page":"10036","article-title":"Montage: An astronomical image mosaicking toolkit","volume":"1","author":"Jacob Joseph C.","year":"2010","unstructured":"Joseph C. Jacob 2010 . Montage: An astronomical image mosaicking toolkit . Astrophys. Source Code Libr. 1 (2010), 10036 . Joseph C. Jacob et al. 2010. Montage: An astronomical image mosaicking toolkit. Astrophys. Source Code Libr. 1 (2010), 10036.","journal-title":"Astrophys. Source Code Libr."},{"key":"e_1_2_1_34_1","unstructured":"Gideon Juve et al. 2017. Synthetic workflow generators. Retrieved from https:\/\/github.com\/pegasus-isi\/WorkflowGenerator.  Gideon Juve et al. 2017. Synthetic workflow generators. Retrieved from https:\/\/github.com\/pegasus-isi\/WorkflowGenerator."},{"volume-title":"Bioinformatic discovery of bacterial regulatory RNAs using SIPHT","author":"Livny Jonathan","key":"e_1_2_1_35_1","unstructured":"Jonathan Livny . 2012. Bioinformatic discovery of bacterial regulatory RNAs using SIPHT . In Bacterial Regulatory RNA. Springer , 3--14. Jonathan Livny. 2012. Bioinformatic discovery of bacterial regulatory RNAs using SIPHT. In Bacterial Regulatory RNA. Springer, 3--14."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807128.1807138"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10723-014-9314-7"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0743-7315(03)00108-4"},{"volume-title":"Proceedings of ACM\/IEEE Conference on Supercomputing.","author":"Maciej","key":"e_1_2_1_39_1","unstructured":"Maciej Malawski et al. 2012. Cost-and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds . In Proceedings of ACM\/IEEE Conference on Supercomputing. Maciej Malawski et al. 2012. Cost-and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. In Proceedings of ACM\/IEEE Conference on Supercomputing."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1155\/2015\/680271"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063384.2063449"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2013.61"},{"volume-title":"Proceedings of IEEE\/ACM CCGrid.","author":"Athanasios","key":"e_1_2_1_43_1","unstructured":"Athanasios Naskos et al. 2015. Dependable horizontal scaling based on probabilistic model checking . In Proceedings of IEEE\/ACM CCGrid. Athanasios Naskos et al. 2015. Dependable horizontal scaling based on probabilistic model checking. In Proceedings of IEEE\/ACM CCGrid."},{"volume-title":"Proceedings of the CoreGRID Integration Workshop.","author":"Simon","key":"e_1_2_1_44_1","unstructured":"Simon Ostermann et al. 2008. On the characteristics of grid workflows . In Proceedings of the CoreGRID Integration Workshop. Simon Ostermann et al. 2008. On the characteristics of grid workflows. In Proceedings of the CoreGRID Integration Workshop."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2930659"},{"volume-title":"Proceedings of IEEE IC2E.","author":"Mayank","key":"e_1_2_1_46_1","unstructured":"Mayank Pundir et al. 2016. Supporting on-demand elasticity in distributed graph processing . In Proceedings of IEEE IC2E. Mayank Pundir et al. 2016. Supporting on-demand elasticity in distributed graph processing. In Proceedings of IEEE IC2E."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2011.243"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.peva.2015.07.005"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2013.162"},{"key":"e_1_2_1_50_1","volume-title":"Taylor et al","author":"Ian","year":"2014","unstructured":"Ian J. Taylor et al . 2014 . Workflows for e-Science: Scientific Workflows for Grids. Springer . Ian J. Taylor et al. 2014. Workflows for e-Science: Scientific Workflows for Grids. Springer."},{"key":"e_1_2_1_51_1","unstructured":"Sachin Tilloo. 2017. Running arbitrary DAG-based workflows in the cloud. Retrieved from http:\/\/www.ebaytechblog.com\/2016\/04\/05\/running-arbitrary-dag-based-workflows-in-the-cloud.  Sachin Tilloo. 2017. Running arbitrary DAG-based workflows in the cloud. Retrieved from http:\/\/www.ebaytechblog.com\/2016\/04\/05\/running-arbitrary-dag-based-workflows-in-the-cloud."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1064212.1064252"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1342171.1342172"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPP.2008.68"},{"key":"e_1_2_1_55_1","unstructured":"Katherine Yelick et al. 2011. The Magellan report on cloud computing for science. U.S. Department of Energy Washington DC Tech. Rep (2011).  Katherine Yelick et al. 2011. The Magellan report on cloud computing for science. U.S. Department of Energy Washington DC Tech. Rep (2011)."}],"container-title":["ACM Transactions on Modeling and Performance Evaluation of Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3164537","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3164537","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3164537","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:13:38Z","timestamp":1750212818000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3164537"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,4,10]]},"references-count":55,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2018,6,30]]}},"alternative-id":["10.1145\/3164537"],"URL":"https:\/\/doi.org\/10.1145\/3164537","relation":{},"ISSN":["2376-3639","2376-3647"],"issn-type":[{"type":"print","value":"2376-3639"},{"type":"electronic","value":"2376-3647"}],"subject":[],"published":{"date-parts":[[2018,4,10]]},"assertion":[{"value":"2017-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-04-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}