{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T10:11:17Z","timestamp":1781086277996,"version":"3.54.1"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2024,3,12]],"date-time":"2024-03-12T00:00:00Z","timestamp":1710201600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research and Development Plan","award":["2021YFB3300500"],"award-info":[{"award-number":["2021YFB3300500"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2024,3,12]]},"abstract":"<jats:p>When analyzing time series, often interactively, the analysts frequently demand to visualize instantly large-scale data stored in databases. M4 visualization selects the first, last, bottom and top data points in each pixel column to ensure pixel-perfectness of the two-color line chart visualization. While M4 already shows its preciseness of encasing time series in different scales into a fixed size of pixels, how to efficiently support M4 representation in a time series native database is still absent. It is worth noting that, to enable fast writes, the commodity time series database systems, such as Apache IoTDB or InfluxDB, employ LSM-Tree based storage. That is, a time series is segmented and stored in a number of chunks, with possibly out-of-order arrivals, i.e., disordered on timestamps. To implement M4, a natural idea is to merge online the chunks as a whole series, with costly merge sort on timestamps, and then perform M4 representation as in relational databases. In this study, we propose a novel chunk merge free approach called M4-LSM to accelerate M4 representation and visualization. In particular, we utilize the metadata of chunks to prune and avoid the costly merging of any chunk. Moreover, intra-chunk indexing and pruning are enabled for efficiently accessing the representation points, referring to the special properties of time series. Remarkably, the time series database native operator M4-LSM has been implemented in Apache IoTDB, an open-source time series database, and deployed in companies across various industries. In the experiments over real-world datasets, the proposed M4-LSM operator demonstrates high efficiency without sacrificing preciseness.<\/jats:p>","DOI":"10.1145\/3639290","type":"journal-article","created":{"date-parts":[[2024,3,26]],"date-time":"2024-03-26T18:51:32Z","timestamp":1711479092000},"page":"1-26","source":"Crossref","is-referenced-by-count":5,"title":["Time Series Representation for Visualization in Apache IoTDB"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-0112-8329","authenticated-orcid":false,"given":"Lei","family":"Rui","sequence":"first","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6868-4045","authenticated-orcid":false,"given":"Xiangdong","family":"Huang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9503-2755","authenticated-orcid":false,"given":"Shaoxu","family":"Song","sequence":"additional","affiliation":[{"name":"BNRist, Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4734-4882","authenticated-orcid":false,"given":"Yuyuan","family":"Kang","sequence":"additional","affiliation":[{"name":"University of Wisconsin-Madison, Madison, WI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1698-8992","authenticated-orcid":false,"given":"Chen","family":"Wang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6841-7943","authenticated-orcid":false,"given":"Jianmin","family":"Wang","sequence":"additional","affiliation":[{"name":"BNRist, Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,3,26]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"https:\/\/cwiki.apache.org\/confluence\/display\/IOTDB\/QueryFundamentals."},{"key":"e_1_2_1_2_1","unstructured":"https:\/\/debs.org\/grand-challenges\/2012\/."},{"key":"e_1_2_1_3_1","unstructured":"https:\/\/github.com\/apache\/iotdb\/tree\/research\/M4-visualization."},{"key":"e_1_2_1_4_1","unstructured":"https:\/\/github.com\/thssdb\/M4-LSM."},{"key":"e_1_2_1_5_1","unstructured":"https:\/\/github.com\/thssdb\/M4-LSM\/blob\/supplement\/supplement.pdf."},{"key":"e_1_2_1_6_1","unstructured":"https:\/\/iotdb.apache.org."},{"key":"e_1_2_1_7_1","unstructured":"https:\/\/iotdb.apache.org\/UserGuide\/V1.1.x\/Operators-Functions\/Sample.html#m4-function."},{"key":"e_1_2_1_8_1","unstructured":"https:\/\/www.iis.fraunhofer.de\/en\/ff\/lv\/dataanalytics\/ek\/download.html."},{"key":"e_1_2_1_9_1","unstructured":"https:\/\/www.influxdata.com\/."},{"key":"e_1_2_1_10_1","unstructured":"https:\/\/www.kaggle.com\/datasets\/xxx123456789\/exp-datasets."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-57301-1_5"},{"key":"e_1_2_1_12_1","first-page":"126","volume-title":"Proceedings of the 15th International Conference on Data Engineering","author":"Chan K.","year":"1999","unstructured":"K. Chan and A. W. Fu. Efficient time series matching by wavelets. In M. Kitsuregawa, M. P. Papazoglou, and C. Pu, editors, Proceedings of the 15th International Conference on Data Engineering, Sydney, Australia, March 23--26, 1999, pages 126--133. IEEE Computer Society, 1999."},{"key":"e_1_2_1_13_1","volume-title":"Synopses for massive data: Samples, histograms, wavelets, sketches. Found. Trends Databases, 4(1--3):1--294","author":"Cormode G.","year":"2012","unstructured":"G. Cormode, M. N. Garofalakis, P. J. Haas, and C. Jermaine. Synopses for massive data: Samples, histograms, wavelets, sketches. Found. Trends Databases, 4(1--3):1--294, 2012."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3064054"},{"key":"e_1_2_1_15_1","first-page":"505","volume-title":"Proceedings of the 2018 International Conference on Management of Data, SIGMOD Conference 2018","author":"Dayan N.","year":"2018","unstructured":"N. Dayan and S. Idreos. Dostoevsky: Better space-time trade-offs for lsm-tree based key-value stores via adaptive removal of superfluous merging. In G. Das, C. M. Jermaine, and P. A. Bernstein, editors, Proceedings of the 2018 International Conference on Management of Data, SIGMOD Conference 2018, Houston, TX, USA, June 10--15, 2018, pages 505--520. ACM, 2018."},{"key":"e_1_2_1_16_1","first-page":"449","volume-title":"Proceedings of the 2019 International Conference on Management of Data, SIGMOD Conference 2019","author":"Dayan N.","year":"2019","unstructured":"N. Dayan and S. Idreos. The log-structured merge-bush & the wacky continuum. In P. A. Boncz, S. Manegold, A. Ailamaki, A. Deshpande, and T. Kraska, editors, Proceedings of the 2019 International Conference on Management of Data, SIGMOD Conference 2019, Amsterdam, The Netherlands, June 30 - July 5, 2019, pages 449--466. ACM, 2019."},{"key":"e_1_2_1_17_1","volume-title":"Data point selection for line chart visualization: Methodological assessment and evidence-based guidelines. CoRR, abs\/2304.00900","author":"Donckt J. V. D.","year":"2023","unstructured":"J. V. D. Donckt, M. J. V. D. Donckt, M. Rademaker, and S. V. Hoecke. Data point selection for line chart visualization: Methodological assessment and evidence-based guidelines. CoRR, abs\/2304.00900, 2023."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2379776.2379788"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.14778\/3538598.3538607"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623613"},{"key":"e_1_2_1_21_1","first-page":"2667","volume-title":"Proceedings of the 2020 International Conference on Management of Data, SIGMOD Conference 2020, online conference [Portland, OR, USA], June 14--19, 2020","author":"Idreos S.","year":"2020","unstructured":"S. Idreos and M. Callaghan. Key-value storage engines. In D. Maier, R. Pottinger, A. Doan, W. Tan, A. Alawini, and H. Q. Ngo, editors, Proceedings of the 2020 International Conference on Management of Data, SIGMOD Conference 2020, online conference [Portland, OR, USA], June 14--19, 2020, pages 2667--2672. ACM, 2020."},{"key":"e_1_2_1_22_1","volume-title":"9th Biennial Conference on Innovative Data Systems Research, CIDR 2019, Asilomar, CA, USA, January 13--16, 2019, Online Proceedings. www.cidrdb.org","author":"Idreos S.","year":"2019","unstructured":"S. Idreos, N. Dayan, W. Qin, M. Akmanalp, S. Hilgard, A. Ross, J. Lennon, V. Jain, H. Gupta, D. Li, and Z. Zhu. Design continuums and the path toward self-designing key-value stores that know and learn. In 9th Biennial Conference on Innovative Data Systems Research, CIDR 2019, Asilomar, CA, USA, January 13--16, 2019, Online Proceedings. www.cidrdb.org, 2019."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2017.2740932"},{"key":"e_1_2_1_24_1","unstructured":"U. Jugel. Visualization-driven data aggregation: rethinking data acquisition for data visualizations. PhD thesis Technical University of Berlin Germany 2017."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732951.2732953"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE53745.2022.00315"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/PL00011669"},{"key":"e_1_2_1_28_1","volume-title":"SIGMOD 1997, Proceedings ACM SIGMOD International Conference on Management of Data, May 13--15, 1997","author":"Korn F.","year":"1997","unstructured":"F. Korn, H. V. Jagadish, and C. Faloutsos. Efficiently supporting ad hoc queries in large datasets of time sequences. In J. Peckham, editor, SIGMOD 1997, Proceedings ACM SIGMOD International Conference on Management of Data, May 13--15, 1997, Tucson, Arizona, USA, pages 289--300. ACM Press, 1997."},{"key":"e_1_2_1_29_1","first-page":"489","volume-title":"Proceedings of the 2018 International Conference on Management of Data, SIGMOD Conference 2018","author":"Kraska T.","year":"2018","unstructured":"T. Kraska, A. Beutel, E. H. Chi, J. Dean, and N. Polyzotis. The case for learned index structures. In G. Das, C. M. Jermaine, and P. A. Bernstein, editors, Proceedings of the 2018 International Conference on Management of Data, SIGMOD Conference 2018, Houston, TX, USA, June 10--15, 2018, pages 489--504. ACM, 2018."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3470827"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.14778\/3384345.3384357"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-007-0064-z"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623741"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346452"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.14778\/3421424.3421425"},{"key":"e_1_2_1_36_1","volume-title":"Visualizing a million time series with the density line chart. CoRR, abs\/1808.06019","author":"Moritz D.","year":"2018","unstructured":"D. Moritz and D. Fisher. Visualizing a million time series with the density line chart. CoRR, abs\/1808.06019, 2018."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s002360050048"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2016.7498287"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1401890.1401966"},{"key":"e_1_2_1_40_1","volume-title":"Downsampling time series for visual representation. Master's thesis","author":"Steinarsson S.","year":"2013","unstructured":"S. Steinarsson. Downsampling time series for visual representation. Master's thesis, University of Iceland, 2013."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/VIS54862.2022.00013"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589775"},{"key":"e_1_2_1_43_1","first-page":"36","volume-title":"Proceedings of the 2nd International Conference on Internet of Things, Big Data and Security, IoTBDS 2017","author":"Weiss W.","year":"2017","unstructured":"W. Weiss, V. J. E. Jim\u00e9nez, and H. Zeiner. A dataset and a comparison of out-of-order event compensation algorithms. In M. Ramachandran, V. M. Mu\u00f1oz, V. Kantere, G. B. Wills, R. J. Walters, and V. Chang, editors, Proceedings of the 2nd International Conference on Internet of Things, Big Data and Security, IoTBDS 2017, Porto, Portugal, April 24--26, 2017, pages 36--46. SciTePress, 2017."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.14778\/3547305.3547319"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11390-022-1573-3"},{"key":"e_1_2_1_46_1","doi-asserted-by":"crossref","unstructured":"L. Ye and E. J. Keogh. Time series shapelets: a new primitive for data mining. In J. F. E. IV F. Fogelman-Souli\u00e9 P. A. Flach and M. J. Zaki editors Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining Paris France June 28 - July 1 2009 pages 947--956. ACM 2009.","DOI":"10.1145\/1557019.1557122"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s41019-023-00205-y"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.14778\/3213880.3213885"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114865"}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3639290","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3639290","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T15:17:21Z","timestamp":1755789441000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3639290"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,12]]},"references-count":49,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,3,12]]}},"alternative-id":["10.1145\/3639290"],"URL":"https:\/\/doi.org\/10.1145\/3639290","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,12]]}}}