{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T22:24:52Z","timestamp":1780439092202,"version":"3.54.1"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"11","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2018,7]]},"abstract":"<jats:p>\n            Industrial systems, e.g., wind turbines, generate big amounts of data from reliable sensors with high velocity. As it is unfeasible to store and query such big amounts of data, only simple aggregates are currently stored. However, aggregates remove fluctuations and outliers that can reveal underlying problems and limit the knowledge to be gained from historical data. As a remedy, we present the distributed Time Series Management System (TSMS)\n            <jats:italic>ModelarDB<\/jats:italic>\n            that uses\n            <jats:italic>models<\/jats:italic>\n            to store sensor data. We thus propose an online, adaptive multi-model compression algorithm that maintains data values within a user-defined error bound (possibly zero). We also propose (i) a database schema to store time series as models, (ii) methods to push-down predicates to a key-value store utilizing this schema, (iii) optimized methods to execute aggregate queries on models, (iv) a method to optimize execution of projections through static code-generation, and (v) dynamic extensibility that allows new models to be used without recompiling the TSMS. Further, we present a general modular distributed TSMS architecture and its implementation, ModelarDB, as a portable library, using Apache Spark for query processing and Apache Cassandra for storage. An experimental evaluation shows that, unlike current systems, ModelarDB hits a sweet spot and offers fast ingestion, good compression, and fast, scalable online aggregate query processing at the same time. This is achieved by dynamically adapting to data sets using multiple models. The system degrades gracefully as more outliers occur and the actual errors are much lower than the bounds.\n          <\/jats:p>","DOI":"10.14778\/3236187.3236215","type":"journal-article","created":{"date-parts":[[2018,9,10]],"date-time":"2018-09-10T12:12:28Z","timestamp":1536581548000},"page":"1688-1701","source":"Crossref","is-referenced-by-count":44,"title":["ModelarDB"],"prefix":"10.14778","volume":"11","author":[{"given":"S\u00f8ren Kejser","family":"Jensen","sequence":"first","affiliation":[{"name":"Aalborg University, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Torben Bach","family":"Pedersen","sequence":"additional","affiliation":[{"name":"Aalborg University, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian","family":"Thomsen","sequence":"additional","affiliation":[{"name":"Aalborg University, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,7]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Apache Cassanddra - Hardware Choices. http:\/\/cassandra.apache.org\/doc\/latest\/operating\/hardware.html. Viewed: 2018-07-15.  Apache Cassanddra - Hardware Choices. http:\/\/cassandra.apache.org\/doc\/latest\/operating\/hardware.html. Viewed: 2018-07-15."},{"key":"e_1_2_1_2_1","unstructured":"Apache Cassandra. http:\/\/cassandra.apache.org\/. Viewed: 2018-07-15.  Apache Cassandra. http:\/\/cassandra.apache.org\/. Viewed: 2018-07-15."},{"key":"e_1_2_1_3_1","unstructured":"Apache Flink. https:\/\/flink.apache.org\/. Viewed: 2018-07-15.  Apache Flink. https:\/\/flink.apache.org\/. Viewed: 2018-07-15."},{"key":"e_1_2_1_4_1","unstructured":"Apache HBase. https:\/\/hbase.apache.org\/. Viewed: 2018-07-15.  Apache HBase. https:\/\/hbase.apache.org\/. Viewed: 2018-07-15."},{"key":"e_1_2_1_5_1","unstructured":"Apache MongoDB. https:\/\/www.mongodb.com\/. Viewed: 2018-07-15.  Apache MongoDB. https:\/\/www.mongodb.com\/. Viewed: 2018-07-15."},{"key":"e_1_2_1_6_1","unstructured":"Apache ORC. https:\/\/orc.apache.org\/. Viewed: 2018-07-15.  Apache ORC. https:\/\/orc.apache.org\/. Viewed: 2018-07-15."},{"key":"e_1_2_1_7_1","unstructured":"Apache Parquet. https:\/\/parquet.apache.org\/. Viewed: 2018-07-15.  Apache Parquet. https:\/\/parquet.apache.org\/. Viewed: 2018-07-15."},{"key":"e_1_2_1_8_1","unstructured":"Apache Spark. https:\/\/spark.apache.org\/. Viewed: 2018-07-15.  Apache Spark. https:\/\/spark.apache.org\/. Viewed: 2018-07-15."},{"key":"e_1_2_1_9_1","unstructured":"Apache Spark - Hardware Provisioning. https:\/\/spark.apache.org\/docs\/2.1.0\/hardware-provisioning.html. Viewed: 2018-07-15.  Apache Spark - Hardware Provisioning. https:\/\/spark.apache.org\/docs\/2.1.0\/hardware-provisioning.html. Viewed: 2018-07-15."},{"key":"e_1_2_1_10_1","unstructured":"Azure Databricks. https:\/\/azure.microsoft.com\/enus\/pricing\/details\/databricks\/. Viewed: 2018-07-15.  Azure Databricks. https:\/\/azure.microsoft.com\/enus\/pricing\/details\/databricks\/. Viewed: 2018-07-15."},{"key":"e_1_2_1_11_1","unstructured":"DB-Engines Ranking. https:\/\/db-engines.com\/en\/ranking. Viewed: 2018-07-15.  DB-Engines Ranking. https:\/\/db-engines.com\/en\/ranking. Viewed: 2018-07-15."},{"key":"e_1_2_1_12_1","unstructured":"DiCyPS - Center for Data-Intensive Cyber-Physical Systems. http:\/\/www.dicyps.dk\/dicyps-in-english\/. Viewed: 2018-07-15.  DiCyPS - Center for Data-Intensive Cyber-Physical Systems. http:\/\/www.dicyps.dk\/dicyps-in-english\/. Viewed: 2018-07-15."},{"key":"e_1_2_1_13_1","unstructured":"InfluxData InfluxDB. https:\/\/www.influxdata.com\/. Viewed: 2018-07-15.  InfluxData InfluxDB. https:\/\/www.influxdata.com\/. Viewed: 2018-07-15."},{"key":"e_1_2_1_14_1","unstructured":"InfluxDB API Client Libraries. https:\/\/docs.influxdata.com\/influxdb\/v1.4\/tools\/api_client_libraries\/. Viewed: 2018-07-15.  InfluxDB API Client Libraries. https:\/\/docs.influxdata.com\/influxdb\/v1.4\/tools\/api_client_libraries\/. Viewed: 2018-07-15."},{"key":"e_1_2_1_15_1","unstructured":"Java Virtual Machine Specification. https:\/\/docs.oracle.com\/javase\/specs\/jvms\/se8\/html\/jvms-3.html#jvms-3.10. Viewed: 2018-07-15.  Java Virtual Machine Specification. https:\/\/docs.oracle.com\/javase\/specs\/jvms\/se8\/html\/jvms-3.html#jvms-3.10. Viewed: 2018-07-15."},{"key":"e_1_2_1_16_1","unstructured":"Microsoft Azure for Research. https:\/\/www.microsoft.com\/enus\/research\/academic-program\/microsoft-azure-for-research\/. Viewed: 2018-07-15.  Microsoft Azure for Research. https:\/\/www.microsoft.com\/enus\/research\/academic-program\/microsoft-azure-for-research\/. Viewed: 2018-07-15."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2742797"},{"issue":"4","key":"e_1_2_1_18_1","first-page":"28","article-title":"Apache Flink: Stream and Batch Processing in a Single Engine","volume":"38","author":"Carbone P.","year":"2015","unstructured":"P. Carbone , A. Katsifodimos , S. Ewen , V. Markl , S. Haridi , and K. Tzoumas . Apache Flink: Stream and Batch Processing in a Single Engine . Bulletin of the IEEE Computer Society Technical Committee on Data Engineering , 38 ( 4 ): 28 -- 38 , 2015 . P. Carbone, A. Katsifodimos, S. Ewen, V. Markl, S. Haridi, and K. Tzoumas. Apache Flink: Stream and Batch Processing in a Single Engine. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 38(4):28--38, 2015.","journal-title":"Bulletin of the IEEE Computer Society Technical Committee on Data Engineering"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807128.1807152"},{"key":"e_1_2_1_20_1","first-page":"137","volume-title":"Proceedings of OSDI","author":"Dean J.","year":"2004","unstructured":"J. Dean and S. Ghemawat . MapReduce: Simplified Data Processing on Large Clusters . In Proceedings of OSDI , pages 137 -- 150 . USENIX, 2004 . J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In Proceedings of OSDI, pages 137--150. USENIX, 2004."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1142473.1142483"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-014-0368-8"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687627.1687645"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.bdr.2014.07.005"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2258056.2258059"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2595630"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2012.237"},{"issue":"11","key":"e_1_2_1_28_1","first-page":"2581","volume":"29","author":"Jensen S. K.","year":"2017","unstructured":"S. K. Jensen , T. B. Pedersen , and C. Thomsen . Time Series Management Systems: A Survey. IEEE TKDE , 29 ( 11 ): 2581 -- 2600 , 2017 . S. K. Jensen, T. B. Pedersen, and C. Thomsen. Time Series Management Systems: A Survey. IEEE TKDE, 29(11):2581--2600, 2017.","journal-title":"Time Series Management Systems: A Survey. IEEE TKDE"},{"key":"e_1_2_1_29_1","volume-title":"Proceedings of CIDR","author":"Katsis Y.","year":"2015","unstructured":"Y. Katsis , Y. Freund , and Y. Papakonstantinou . Combining Databases and Signal Processing in Plato . In Proceedings of CIDR , 2015 . Y. Katsis, Y. Freund, and Y. Papakonstantinou. Combining Databases and Signal Processing in Plato. In Proceedings of CIDR, 2015."},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of SustKDD","author":"Kolter J. Z.","year":"2011","unstructured":"J. Z. Kolter and M. J. Johnson . REDD: A Public Data Set for Energy Disaggregation Research . In Proceedings of SustKDD , 2011 . J. Z. Kolter and M. J. Johnson. REDD: A Public Data Set for Energy Disaggregation Research. In Proceedings of SustKDD, 2011."},{"key":"e_1_2_1_31_1","first-page":"538","volume-title":"Proceedings of IJCAI","author":"Korf R. E.","year":"2009","unstructured":"R. E. Korf . Multi-Way Number Partitioning . In Proceedings of IJCAI , pages 538 -- 543 , 2009 . R. E. Korf. Multi-Way Number Partitioning. In Proceedings of IJCAI, pages 538--543, 2009."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1773912.1773922"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2003.1260811"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2015.7113282"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2013.6691772"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2014.7004244"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/MDM.2011.57"},{"key":"e_1_2_1_38_1","volume-title":"EPFL LSIR","author":"Papaioannou T. G.","year":"2011","unstructured":"T. G. Papaioannou , M. Riahi , and K. Aberer . Towards Online Multi-Model Approximation of Time Series. Technical report , EPFL LSIR , 2011 . T. G. Papaioannou, M. Riahi, and K. Aberer. Towards Online Multi-Model Approximation of Time Series. Technical report, EPFL LSIR, 2011."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824078"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11280-013-0256-y"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-6309-2_2"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSST.2010.5496972"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1376616.1376696"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2915237"},{"key":"e_1_2_1_45_1","volume-title":"Proceedings of HotCloud. USENIX","author":"Zaharia M.","year":"2010","unstructured":"M. Zaharia , M. Chowdhury , M. J. Franklin , S. Shenker , and I. Stoica . Spark: Cluster Computing with Working Sets . In Proceedings of HotCloud. USENIX , 2010 . M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica. Spark: Cluster Computing with Working Sets. In Proceedings of HotCloud. USENIX, 2010."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2517349.2522737"},{"key":"e_1_2_1_47_1","volume-title":"Proceedings of HotCloud. USENIX","author":"Zaharia M.","year":"2012","unstructured":"M. Zaharia , T. Das , H. Li , S. Shenker , and I. Stoica . Discretized Streams: An Efficient and Fault-Tolerant Model for Stream Processing on Large Clusters . In Proceedings of HotCloud. USENIX , 2012 . M. Zaharia, T. Das, H. Li, S. Shenker, and I. Stoica. Discretized Streams: An Efficient and Fault-Tolerant Model for Stream Processing on Large Clusters. In Proceedings of HotCloud. USENIX, 2012."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3236187.3236215","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T09:44:03Z","timestamp":1672220643000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3236187.3236215"}},"subtitle":["modular model-based time series management with spark and cassandra"],"short-title":[],"issued":{"date-parts":[[2018,7]]},"references-count":47,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2018,7]]}},"alternative-id":["10.14778\/3236187.3236215"],"URL":"https:\/\/doi.org\/10.14778\/3236187.3236215","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2018,7]]}}}