{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,14]],"date-time":"2025-05-14T03:49:01Z","timestamp":1747194541734,"version":"3.40.5"},"reference-count":19,"publisher":"SAGE Publications","issue":"7","license":[{"start":{"date-parts":[[2019,7,1]],"date-time":"2019-07-01T00:00:00Z","timestamp":1561939200000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["International Journal of Distributed Sensor Networks"],"published-print":{"date-parts":[[2019,7]]},"abstract":"<jats:p> In this article, we address dynamic workflow management for sampling and filtering data streams in Apache Storm. As many sensors generate data streams continuously, we often use sampling to choose some representative data or filtering to remove unnecessary data. Apache Storm is a real-time distributed processing platform suitable for handling large data streams. Storm, however, must stop the entire work when it changes the input data structure or processing algorithm as it needs to modify, redistribute, and restart the programs. In addition, for effective data processing, we often use Storm with Kafka and databases, but it is difficult to use these platforms in an integrated manner. In this article, we derive the problems when applying sampling and filtering algorithms to Storm and propose a dynamic workflow management model that solves these problems. First, we present the concept of a plan consisting of input, processing, and output modules of a data stream. Second, we propose Storm Plan Manager, which can operate Storm, Kafka, and database as a single integrated system. Storm Plan Manager is an integrated workflow manager that dynamically controls sampling and filtering of data streams through plans. Third, as a key feature, Storm Plan Manager provides a Web client interface to visually create, execute, and monitor plans. In this article, we show the usefulness of the proposed Storm Plan Manager by presenting its design, implementation, and experimental results in order. <\/jats:p>","DOI":"10.1177\/1550147719862206","type":"journal-article","created":{"date-parts":[[2019,7,10]],"date-time":"2019-07-10T02:11:01Z","timestamp":1562724661000},"page":"155014771986220","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":3,"title":["SPMgr: Dynamic workflow manager for sampling and filtering data streams over Apache Storm"],"prefix":"10.1177","volume":"15","author":[{"given":"Youngkuk","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Computer Science, Kangwon National University, Chuncheon, South Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Siwoon","family":"Son","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Kangwon National University, Chuncheon, South Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2396-0405","authenticated-orcid":false,"given":"Yang-Sae","family":"Moon","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Kangwon National University, Chuncheon, South Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2019,7,9]]},"reference":[{"first-page":"1","volume-title":"Proceedings of the 21st ACM symposium on principles of database systems","author":"Babcock B","key":"bibr1-1550147719862206"},{"first-page":"1975","volume-title":"Proceedings of the 20th international conference on knowledge discovery and data mining","author":"Cormode G","key":"bibr2-1550147719862206"},{"first-page":"1","volume-title":"Proceedings of the international conference on management of data, ACM SIGMOD","author":"Johnson T","key":"bibr3-1550147719862206"},{"key":"bibr4-1550147719862206","unstructured":"Manyika J, Chui M, Brown B, et al. Big data: the next frontier for innovation, competition, and productivity. Technical report, McKinsey Global Institute, New York, May 2011."},{"first-page":"147","volume-title":"Proceedings of the international conference on management of data, ACM SIGMOD","author":"Toshniwal A","key":"bibr6-1550147719862206"},{"first-page":"1","volume-title":"Proceedings of the NetDB","author":"Kreps J","key":"bibr9-1550147719862206"},{"key":"bibr15-1550147719862206","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-28608-0_2"},{"key":"bibr16-1550147719862206","volume-title":"Sampling techniques","author":"Cochran WG","year":"1977","edition":"3"},{"volume-title":"Mining of massive datasets","year":"2015","author":"Leskovec J","key":"bibr17-1550147719862206"},{"first-page":"29","volume-title":"Proceedings of international conference on digital economy","author":"Sibai RE","key":"bibr18-1550147719862206"},{"key":"bibr19-1550147719862206","doi-asserted-by":"publisher","DOI":"10.1145\/3147.3165"},{"first-page":"159","volume-title":"Proceedings of the 21st international conference on knowledge discovery and data mining","author":"Cohen E","key":"bibr20-1550147719862206"},{"first-page":"77","volume-title":"Proceedings of the 21st ACM symposium on principles of database systems","author":"Cormode G","key":"bibr21-1550147719862206"},{"issue":"4","key":"bibr22-1550147719862206","first-page":"1","volume":"14","author":"Kim H","year":"2018","journal-title":"Int J Distrib Sens N"},{"issue":"11","key":"bibr23-1550147719862206","first-page":"1","volume":"14","author":"Cho W","year":"2018","journal-title":"Int J Distrib Sens N"},{"key":"bibr24-1550147719862206","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2009.63"},{"first-page":"101","volume-title":"Proceedings of the IEEE international conference on semantic computing","author":"Shin J","key":"bibr25-1550147719862206"},{"key":"bibr26-1550147719862206","doi-asserted-by":"publisher","DOI":"10.3390\/s19051168"},{"first-page":"5492","volume-title":"Proceedings of the IEEE international conference on decision and control","author":"Olfati-Saber R","key":"bibr27-1550147719862206"}],"container-title":["International Journal of Distributed Sensor Networks"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1550147719862206","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/1550147719862206","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1550147719862206","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,7,10]],"date-time":"2019-07-10T02:11:05Z","timestamp":1562724665000},"score":1,"resource":{"primary":{"URL":"http:\/\/journals.sagepub.com\/doi\/10.1177\/1550147719862206"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7]]},"references-count":19,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2019,7,9]]},"published-print":{"date-parts":[[2019,7]]}},"alternative-id":["10.1177\/1550147719862206"],"URL":"https:\/\/doi.org\/10.1177\/1550147719862206","relation":{},"ISSN":["1550-1477","1550-1477"],"issn-type":[{"type":"print","value":"1550-1477"},{"type":"electronic","value":"1550-1477"}],"subject":[],"published":{"date-parts":[[2019,7]]}}}