{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,3]],"date-time":"2025-05-03T13:10:02Z","timestamp":1746277802676,"version":"3.40.4"},"reference-count":28,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2012,8]]},"abstract":"<jats:p>Stream processing applications are deployed as continuous queries that run from the time of their submission until their cancellation. This deployment mode limits developers who need their applications to perform runtime adaptation, such as algorithmic adjustments, incremental job deployment, and application-specific failure recovery. Currently, developers do runtime adaptation by using external scripts and\/or by inserting operators into the stream processing graph that are unrelated to the data processing logic. In this paper, we describe a component called<jats:italic>orchestrator<\/jats:italic>that allows users to write routines for automatically adapting the application to runtime conditions. Developers build an orchestrator by registering and handling events as well as specifying actuations. Events can be generated due to changes in the system state (e.g., application component failures), built-in system metrics (e.g., throughput of a connection), or custom application metrics (e.g., quality score). Once the orchestrator receives an event, users can take adaptation actions by using the orchestrator actuation APIs. We demonstrate the use of the orchestrator in IBM's System S in the context of three different applications, illustrating application adaptation to changes on the incoming data distribution, to application failures, and on-demand dynamic composition.<\/jats:p>","DOI":"10.14778\/2367502.2367521","type":"journal-article","created":{"date-parts":[[2014,6,24]],"date-time":"2014-06-24T12:17:57Z","timestamp":1403612277000},"page":"1826-1837","source":"Crossref","is-referenced-by-count":10,"title":["Building user-defined runtime adaptation routines for stream processing applications"],"prefix":"10.14778","volume":"5","author":[{"given":"Gabriela","family":"Jacques-Silva","sequence":"first","affiliation":[{"name":"Thomas J. Watson Research Center, IBM Research, Hawthorne, NY"}]},{"given":"Bu\u011fra","family":"Gedik","sequence":"additional","affiliation":[{"name":"Thomas J. Watson Research Center, IBM Research, Hawthorne, NY and Bilkent University, Ankara, Turkey"}]},{"given":"Rohit","family":"Wagle","sequence":"additional","affiliation":[{"name":"Thomas J. Watson Research Center, IBM Research, Hawthorne, NY"}]},{"given":"Kun-Lung","family":"Wu","sequence":"additional","affiliation":[{"name":"Thomas J. Watson Research Center, IBM Research, Hawthorne, NY"}]},{"given":"Vibhore","family":"Kumar","sequence":"additional","affiliation":[{"name":"Thomas J. Watson Research Center, IBM Research, Hawthorne, NY"}]}],"member":"320","published-online":{"date-parts":[[2012,8]]},"reference":[{"key":"e_1_2_1_1_1","first-page":"277","volume-title":"Proc. of CIDR","author":"Abadi D. J.","year":"2005"},{"issue":"2","key":"e_1_2_1_2_1","first-page":"69","article-title":"Spatio-temporal stream processing Microsoft StreamInsight","volume":"33","author":"Ali M. H.","year":"2010","journal-title":"IEEE Data Engineering Bulletin"},{"key":"e_1_2_1_3_1","unstructured":"Amazon. Amazon simple workflow service. http:\/\/aws.amazon.com\/swf 2012. Amazon. Amazon simple workflow service. http:\/\/aws.amazon.com\/swf 2012."},{"key":"e_1_2_1_4_1","first-page":"27","volume-title":"Proc. of DM-SSP","author":"Amini L.","year":"2006"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-004-0147-z"},{"key":"e_1_2_1_6_1","unstructured":"E. Cerami. Web Services Essentials. O'Reilly Media 2002. E. Cerami. Web Services Essentials . O'Reilly Media 2002."},{"key":"e_1_2_1_7_1","first-page":"875","volume-title":"Proc. of ISCC","author":"Cugola G.","year":"2009"},{"volume-title":"Springer","year":"2006","author":"Doney C.","key":"e_1_2_1_8_1"},{"key":"e_1_2_1_9_1","first-page":"37","volume-title":"Proc. of COORDINATION","author":"Frey D.","year":"2007"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmcj.2005.07.003"},{"key":"e_1_2_1_12_1","first-page":"176","volume-title":"Proc. of ICDE","author":"Hwang J.-H.","year":"2007"},{"key":"e_1_2_1_13_1","unstructured":"IBM. IBM InfoSphere BigInsights. http:\/\/www-01.ibm.com\/software\/data\/infosphere\/biginsights 2011. IBM. IBM InfoSphere BigInsights. http:\/\/www-01.ibm.com\/software\/data\/infosphere\/biginsights 2011."},{"key":"e_1_2_1_14_1","first-page":"641","volume-title":"Proc. of IPDPS","author":"Isert C.","year":"2000"},{"key":"e_1_2_1_15_1","first-page":"145","volume-title":"Proc. of DSN","author":"Jacques-Silva G.","year":"2009"},{"key":"e_1_2_1_16_1","first-page":"231","volume-title":"Proc. of DEBS","author":"Jacques-Silva G.","year":"2011"},{"volume-title":"Proc. of the VLDB Endowment (PVLDB), 1(2):1379--1390","year":"2008","author":"Jain N.","key":"e_1_2_1_17_1"},{"key":"e_1_2_1_18_1","first-page":"308","volume-title":"Proc. of Middleware","author":"Khandekar R.","year":"2009"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1519103.1519105"},{"volume-title":"IBM Redbooks","year":"2006","author":"Manoel E.","key":"e_1_2_1_20_1"},{"key":"e_1_2_1_21_1","unstructured":"MIT. StreamIt language specification version 2.1. http:\/\/groups.csail.mit.edu\/cag\/streamit\/papers\/streamit-lang-spec.pdf 2006. MIT. StreamIt language specification version 2.1. http:\/\/groups.csail.mit.edu\/cag\/streamit\/papers\/streamit-lang-spec.pdf 2006."},{"key":"e_1_2_1_22_1","unstructured":"OASIS Committee. Web services business process execution language (WS-BPEL) version 2.0. http:\/\/docs.oasis-open.org\/wsbpel\/2.0\/wsbpel-v2.0.html 2007. OASIS Committee. Web services business process execution language (WS-BPEL) version 2.0. http:\/\/docs.oasis-open.org\/wsbpel\/2.0\/wsbpel-v2.0.html 2007."},{"key":"e_1_2_1_23_1","first-page":"143","volume-title":"Proc. of Middleware","author":"Ranganathan A.","year":"2003"},{"key":"e_1_2_1_24_1","first-page":"793","volume-title":"Proc. of SIGMOD 2006","author":"Schnaitter K.","year":"2006"},{"key":"e_1_2_1_25_1","first-page":"159","volume-title":"Proc. of VLDB","author":"Tatbul N.","year":"2007"},{"key":"e_1_2_1_26_1","unstructured":"T. White. Hadoop: The Definitive Guide. O'Reilly Media 2009. T. White. Hadoop: The Definitive Guide . O'Reilly Media 2009."},{"key":"e_1_2_1_27_1","first-page":"1185","volume-title":"Proc. of VLDB","author":"Wu K.-L.","year":"2007"},{"key":"e_1_2_1_28_1","doi-asserted-by":"crossref","unstructured":"Yahoo. Oozie workflow engine for Hadoop. http:\/\/yahoo.github.com\/oozie\/releases\/3.1.0\/index.html 2011. Yahoo. Oozie workflow engine for Hadoop. http:\/\/yahoo.github.com\/oozie\/releases\/3.1.0\/index.html 2011.","DOI":"10.1145\/2443416.2443420"},{"key":"e_1_2_1_29_1","first-page":"1087","volume-title":"Proc. of VLDB","author":"Zilio D. C.","year":"2004"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/2367502.2367521","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,3]],"date-time":"2025-05-03T12:53:44Z","timestamp":1746276824000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/2367502.2367521"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,8]]},"references-count":28,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2012,8]]}},"alternative-id":["10.14778\/2367502.2367521"],"URL":"https:\/\/doi.org\/10.14778\/2367502.2367521","relation":{},"ISSN":["2150-8097"],"issn-type":[{"type":"print","value":"2150-8097"}],"subject":[],"published":{"date-parts":[[2012,8]]}}}