{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,26]],"date-time":"2025-11-26T16:26:51Z","timestamp":1764174411604,"version":"3.41.0"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2017,11,14]],"date-time":"2017-11-14T00:00:00Z","timestamp":1510617600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Auton. Adapt. Syst."],"published-print":{"date-parts":[[2017,12,31]]},"abstract":"<jats:p>Data stream management systems (DSMSs) are scalable, highly available, and fault-tolerant systems that aggregate and analyze real-time data in motion. To continuously perform analytics on the fly within the stream, state-of-the-art DSMSs host streaming applications as a set of interconnected operators, with each operator encapsulating the semantic of a specific operation. For parallel execution on a particular platform, these operators need to be appropriately replicated in multiple instances that split and process the workload simultaneously. Because the way operators are partitioned affects the resulting performance of streaming applications, it is essential for DSMSs to have a method to compare different operators and make holistic replication decisions to avoid performance bottlenecks and resource wastage. To this end, we propose a stepwise profiling approach to optimize application performance on a given execution platform. It automatically scales distributed computations over streams based on application features and processing power of provisioned resources and builds the relationship between provisioned resources and application performance metrics to evaluate the efficiency of the resulting configuration. Experimental results confirm that the proposed approach successfully fulfills its goals with minimal profiling overhead.<\/jats:p>","DOI":"10.1145\/3132618","type":"journal-article","created":{"date-parts":[[2017,11,15]],"date-time":"2017-11-15T13:36:32Z","timestamp":1510752992000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":21,"title":["A Stepwise Auto-Profiling Method for Performance Optimization of Streaming Applications"],"prefix":"10.1145","volume":"12","author":[{"given":"Xunyun","family":"Liu","sequence":"first","affiliation":[{"name":"University of Melbourne, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Amir Vahid","family":"Dastjerdi","sequence":"additional","affiliation":[{"name":"PwC Australia, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rodrigo N.","family":"Calheiros","sequence":"additional","affiliation":[{"name":"Western Sydney University, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chenhao","family":"Qu","sequence":"additional","affiliation":[{"name":"University of Melbourne, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rajkumar","family":"Buyya","sequence":"additional","affiliation":[{"name":"University of Melbourne, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2017,11,14]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-003-0095-z"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2006.13"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2488222.2488267"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1869459.1869469"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/UCC.2014.46"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDEW.2007.4401049"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675743.2776766"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPCSim.2016.7568388"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2465282"},{"volume-title":"Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data (SIGMOD\u201903)","author":"Chandrasekaran Sirish","key":"e_1_2_1_10_1","unstructured":"Sirish Chandrasekaran , Owen Cooper , Amol Deshpande , Michael J. Franklin , Joseph M. Hellerstein , Wei Hong , Sailesh Krishnamurthy , Samuel R. Madden , Fred Reiss , and Mehul A. Shah . 2003. TelegraphCQ: Continuous dataflow processing . In Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data (SIGMOD\u201903) . ACM, 668--668. Sirish Chandrasekaran, Owen Cooper, Amol Deshpande, Michael J. Franklin, Joseph M. Hellerstein, Wei Hong, Sailesh Krishnamurthy, Samuel R. Madden, Fred Reiss, and Mehul A. Shah. 2003. TelegraphCQ: Continuous dataflow processing. In Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data (SIGMOD\u201903). ACM, 668--668."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661829.2661882"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/342009.335432"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2670979.2670995"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/CLOUD.2011.75"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2015.7363749"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER.2015.13"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2015.49"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2013.295"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/605397.605428"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2012.24"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2611286.2611309"},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems (DEBS\u201914)","author":"Heinze Thomas","year":"2014","unstructured":"Thomas Heinze , Valerio Pappalardo , Zbigniew Jerzak , and Christof Fetzer . 2014 b. Auto-scaling techniques for elastic data stream processing . In Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems (DEBS\u201914) . ACM, 318--321. Thomas Heinze, Valerio Pappalardo, Zbigniew Jerzak, and Christof Fetzer. 2014b. Auto-scaling techniques for elastic data stream processing. In Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems (DEBS\u201914). ACM, 318--321."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2806777.2806847"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2016.06.010"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/PACT.2009.39"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1100"},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD\u201906)","author":"Jain Navendu","year":"2006","unstructured":"Navendu Jain , Lisa Amini , Henrique Andrade , Richard King , Yoonho Park , Philippe Selo , and Chitra Venkatramani . 2006 . Design, implementation, and evaluation of the linear road benchmark on the stream processing core . In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD\u201906) . ACM, 431--442. Navendu Jain, Lisa Amini, Henrique Andrade, Richard King, Yoonho Park, Philippe Selo, and Chitra Venkatramani. 2006. Design, implementation, and evaluation of the linear road benchmark on the stream processing core. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD\u201906). ACM, 431--442."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2003.1160055"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2742788"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2015.7363773"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2015.48"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10586-013-0281-8"},{"key":"e_1_2_1_33_1","volume-title":"Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems (DEBS\u201916)","author":"Skat Madsen Kasper Grud","year":"2016","unstructured":"Kasper Grud Skat Madsen , Yongluan Zhou , and Li Su . 2016 . Enorm: Efficient window-based computation in large-scale distributed stream processing systems . In Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems (DEBS\u201916) . ACM, 37--48. Kasper Grud Skat Madsen, Yongluan Zhou, and Li Su. 2016. Enorm: Efficient window-based computation in large-scale distributed stream processing systems. In Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems (DEBS\u201916). ACM, 37--48."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2016.08.037"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDEW.2012.38"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2814576.2814808"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1999995.2000026"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2465351.2465353"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3030207.3030227"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2009.5161036"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2370816.2370826"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2771783.2771816"},{"volume-title":"Proceedings of the 19th International Conference on Parallel Architectures and Compilation Techniques (PACT\u201910)","author":"Suleman Muhammad Aater","key":"e_1_2_1_43_1","unstructured":"Muhammad Aater Suleman , Moinuddin K. Qureshi , Khubaib, and Yale N. Patt . 2010. Feedback-directed pipeline parallelism . In Proceedings of the 19th International Conference on Parallel Architectures and Compilation Techniques (PACT\u201910) . ACM, 147--156. Muhammad Aater Suleman, Moinuddin K. Qureshi, Khubaib, and Yale N. Patt. 2010. Feedback-directed pipeline parallelism. In Proceedings of the 19th International Conference on Parallel Architectures and Compilation Techniques (PACT\u201910). ACM, 147--156."},{"key":"e_1_2_1_44_1","first-page":"53","article-title":"Optimizing data stream graph for big data stream computing in cloud datacenter environments","volume":"6","author":"Sun Dawei","year":"2014","unstructured":"Dawei Sun , Ge Fu , Xinran Liu , and Hong Zhang . 2014 . Optimizing data stream graph for big data stream computing in cloud datacenter environments . Int. J. Adv. Comput. Technol. 6 , 5 (2014), 53 -- 65 . Dawei Sun, Ge Fu, Xinran Liu, and Hong Zhang. 2014. Optimizing data stream graph for big data stream computing in cloud datacenter environments. Int. J. Adv. Comput. Technol. 6, 5 (2014), 53--65.","journal-title":"Int. J. Adv. Comput. Technol."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2015.03.027"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/844128.844151"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jnca.2014.09.018"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.5555\/1496950.1496970"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2015.7113328"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2005.53"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2014.61"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/IC2E.2016.38"}],"container-title":["ACM Transactions on Autonomous and Adaptive Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3132618","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3132618","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:10:56Z","timestamp":1750212656000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3132618"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,11,14]]},"references-count":52,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2017,12,31]]}},"alternative-id":["10.1145\/3132618"],"URL":"https:\/\/doi.org\/10.1145\/3132618","relation":{},"ISSN":["1556-4665","1556-4703"],"issn-type":[{"type":"print","value":"1556-4665"},{"type":"electronic","value":"1556-4703"}],"subject":[],"published":{"date-parts":[[2017,11,14]]},"assertion":[{"value":"2016-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-11-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}