{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,13]],"date-time":"2026-08-13T15:58:26Z","timestamp":1786636706805,"version":"build-2736575974"},"reference-count":51,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,5,29]],"date-time":"2024-05-29T00:00:00Z","timestamp":1716940800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["No. 2021YFB3300502"],"award-info":[{"award-number":["No. 2021YFB3300502"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2024,5,29]]},"abstract":"<jats:p>Dirty data are prevalent in time series, such as energy consumption or stock data. Existing data cleaning algorithms present shortcomings in dirty data identification and unsatisfactory cleaning decisions. To handle these drawbacks, we leverage inherent recurrent patterns in time series, analogize them as fixed combinations in textual data, and incorporate the concept of perplexity. The cleaning problem is thus transformed to minimize the perplexity of the time series under a given cleaning cost, and we design a four-phase algorithmic framework to tackle this problem. To ensure the framework's feasibility, we also conduct a brief analysis of the impact of dirty data and devise an automatic budget selection strategy. Moreover, to make it more generic, we additionally introduce advanced solutions, including an ameliorative probability calculation method grounded in the homomorphic pattern aggregation and a greedy-based heuristic algorithm for resource savings. Experiments on 12 real-world datasets demonstrate the superiority of our methods.<\/jats:p>","DOI":"10.1145\/3654993","type":"journal-article","created":{"date-parts":[[2024,5,30]],"date-time":"2024-05-30T09:44:53Z","timestamp":1717062293000},"page":"1-26","source":"Crossref","is-referenced-by-count":6,"title":["Akane: Perplexity-Guided Time Series Data Cleaning"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-6658-8287","authenticated-orcid":false,"given":"Xiaoyu","family":"Han","sequence":"first","affiliation":[{"name":"School of Computer Science, Fudan University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-6386-6571","authenticated-orcid":false,"given":"Haoran","family":"Xiong","sequence":"additional","affiliation":[{"name":"School of Computer Science, Fudan University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2926-4814","authenticated-orcid":false,"given":"Zhenying","family":"He","sequence":"additional","affiliation":[{"name":"School of Computer Science, Fudan University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8136-9621","authenticated-orcid":false,"given":"Peng","family":"Wang","sequence":"additional","affiliation":[{"name":"Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1698-8992","authenticated-orcid":false,"given":"Chen","family":"Wang","sequence":"additional","affiliation":[{"name":"National Engineering Research Center for Big Data Software, EIRI, Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9059-3713","authenticated-orcid":false,"given":"X. Sean","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science, Fudan University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,5,30]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Adam Aboode. 2018. Anomaly detection in time series data based on holt-winters method."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2017.04.070"},{"key":"e_1_2_2_3_1","unstructured":"Arthur Asuncion and David Newman. 2007. UCI machine learning repository."},{"key":"e_1_2_2_4_1","volume-title":"Learning-Based Cleansing for Indoor RFID Data. In SIGMOD Conference. ACM, 925--936","author":"Baba Asif Iqbal","year":"2016","unstructured":"Asif Iqbal Baba, Manfred Jaeger, Hua Lu, Torben Bach Pedersen, Wei-Shinn Ku, and Xike Xie. 2016. Learning-Based Cleansing for Indoor RFID Data. In SIGMOD Conference. ACM, 925--936."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1066157.1066175"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/342009.335388"},{"key":"e_1_2_2_7_1","volume-title":"Time series - data analysis and theory. Classics in applied mathematics","author":"Brillinger David R.","unstructured":"David R. Brillinger. 2001. Time series - data analysis and theory. Classics in applied mathematics, Vol. 36. SIAM."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1051\/ita\/2011106"},{"key":"e_1_2_2_9_1","unstructured":"Stanley F Chen Douglas Beeferman and Roni Rosenfeld. 1998. Evaluation metrics for language models. (1998)."},{"key":"e_1_2_2_10_1","volume-title":"Markov chains. Models, algorithms and applications","author":"Ching Wai-Ki","year":"2006","unstructured":"Wai-Ki Ching and Michael K Ng. 2006. Markov chains. Models, algorithms and applications (2006)."},{"key":"e_1_2_2_11_1","volume-title":"Holistic data cleaning: Putting violations into context","author":"Chu Xu","unstructured":"Xu Chu, Ihab F. Ilyas, and Paolo Papotti. 2013. Holistic data cleaning: Putting violations into context. In ICDE. IEEE Computer Society, 458--469."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2749431"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/72.279188"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00373-011-1026-1"},{"key":"e_1_2_2_15_1","volume-title":"ECML\/PKDD (1) (Lecture Notes in Computer Science","author":"Feremans Len","unstructured":"Len Feremans, Vincent Vercruyssen, Boris Cule, Wannes Meert, and Bart Goethals. 2019. Pattern-Based Anomaly Detection in Mixed-Type Time Series. In ECML\/PKDD (1) (Lecture Notes in Computer Science, Vol. 11906). Springer, 240--256."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.21655\/ijsi.1673-7288.00239"},{"key":"e_1_2_2_17_1","doi-asserted-by":"crossref","unstructured":"Ge Gao Qitong Gao Xi Yang Miroslav Pajic and Min Chi. 2022. A Reinforcement Learning-Informed Pattern Mining Framework for Multivariate Time Series Classification. In IJCAI. ijcai.org 2994--3000.","DOI":"10.24963\/ijcai.2022\/415"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijforecast.2006.03.005"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687627.1687693"},{"key":"e_1_2_2_20_1","volume-title":"Avery","author":"Heim Niklas","year":"2019","unstructured":"Niklas Heim and James E. Avery. 2019. Adaptive Anomaly Detection in Chaotic Time Series with a Spatially Aware Echo State Network. CoRR , Vol. abs\/1909.01709 (2019)."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.envsoft.2009.08.010"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.14778\/3494124.3494147"},{"key":"e_1_2_2_23_1","volume-title":"Mining Seasonal Temporal Patterns in Time Series","author":"Ho Van Long","unstructured":"Van Long Ho, Nguyen Ho, and Torben Bach Pedersen. 2023. Mining Seasonal Temporal Patterns in Time Series. In ICDE. IEEE, 2249--2261."},{"key":"e_1_2_2_24_1","volume-title":"ECML\/PKDD (2) (Lecture Notes in Computer Science","author":"Hooi Bryan","unstructured":"Bryan Hooi, Shenghua Liu, Asim Smailagic, and Christos Faloutsos. 2017. BeatLex: Summarizing and Forecasting Time Series with Patterns. In ECML\/PKDD (2) (Lecture Notes in Computer Science, Vol. 10535). Springer, 3--19."},{"key":"e_1_2_2_25_1","doi-asserted-by":"crossref","unstructured":"Kyle Hundman Valentino Constantinou Christopher Laporte Ian Colwell and Tom S\u00f6 derstr\u00f6 m. 2018. Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding. In KDD. ACM 387--395.","DOI":"10.1145\/3219819.3219845"},{"key":"e_1_2_2_26_1","doi-asserted-by":"crossref","unstructured":"Alexander Ihler Jon Hutchins and Padhraic Smyth. 2006. Adaptive event detection with time-varying poisson processes. In KDD. ACM 207--216.","DOI":"10.1145\/1150402.1150428"},{"key":"e_1_2_2_27_1","volume-title":"Franklin","author":"Jeffery Shawn R.","year":"2006","unstructured":"Shawn R. Jeffery, Minos N. Garofalakis, and Michael J. Franklin. 2006. Adaptive Cleaning for RFID Data Streams. In VLDB. ACM, 163--174."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1121\/1.2016299"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jobe.2020.102110"},{"key":"e_1_2_2_30_1","volume-title":"CleanML: A Study for Evaluating the Impact of Data Cleaning on ML Classification Tasks","author":"Li Peng","unstructured":"Peng Li, Xi Rao, Jennifer Blase, Yue Zhang, Xu Chu, and Ce Zhang. 2021. CleanML: A Study for Evaluating the Impact of Data Cleaning on ML Classification Tasks. In ICDE. IEEE, 13--24."},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.14778\/2535568.2448943"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-007-0064-z"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1002\/jnm.2816"},{"key":"e_1_2_2_34_1","volume-title":"Rokne","author":"Rasheed Faraz","year":"2009","unstructured":"Faraz Rasheed, Peter Peng, Reda Alhajj, and Jon G. Rokne. 2009. Fourier Transform Based Spatial Outlier Mining. In IDEAL (Lecture Notes in Computer Science, Vol. 5788). Springer, 317--324."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137628.3137631"},{"key":"e_1_2_2_36_1","doi-asserted-by":"crossref","unstructured":"Hansheng Ren Bixiong Xu Yujing Wang Chao Yi Congrui Huang Xiaoyu Kou Tony Xing Mao Yang Jie Tong and Qi Zhang. 2019. Time-Series Anomaly Detection Service at Microsoft. In KDD. ACM 3009--3017.","DOI":"10.1145\/3292500.3330680"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.14778\/3538598.3538602"},{"key":"e_1_2_2_38_1","unstructured":"Pavel Senin Jessica Lin Xing Wang Tim Oates Sunil Gandhi Arnold P. Boedihardjo Crystal Chen and Susan Frankenstein. 2015. Time series anomaly discovery with grammar-based compression. In EDBT. OpenProceedings.org 481--492."},{"key":"e_1_2_2_39_1","doi-asserted-by":"crossref","unstructured":"Shaoxu Song Chunping Li and Xiaoquan Zhang. 2015a. Turn Waste into Wealth: On Simultaneous Clustering and Cleaning over Dirty Data. In KDD. ACM 1115--1124.","DOI":"10.1145\/2783258.2783317"},{"key":"e_1_2_2_40_1","doi-asserted-by":"crossref","unstructured":"Shaoxu Song and Aoqian Zhang. 2020. IoT Data Quality. In CIKM. ACM 3517--3518.","DOI":"10.1145\/3340531.3412173"},{"key":"e_1_2_2_41_1","volume-title":"SIGMOD Conference. ACM, 827--841","author":"Song Shaoxu","unstructured":"Shaoxu Song, Aoqian Zhang, Jianmin Wang, and Philip S. Yu. 2015b. SCREEN: Stream Data Cleaning under Speed Constraints. In SIGMOD Conference. ACM, 827--841."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588939"},{"key":"e_1_2_2_43_1","volume-title":"Swapping Repair for Misplaced Attribute Values","author":"Sun Yu","unstructured":"Yu Sun, Shaoxu Song, Chen Wang, and Jianmin Wang. 2020. Swapping Repair for Misplaced Attribute Values. In ICDE. IEEE, 721--732."},{"key":"e_1_2_2_44_1","volume-title":"Topics in Computational Geometry. Ph.,D. Dissertation","author":"Supowit Kenneth J.","unstructured":"Kenneth J. Supowit. 1981. Topics in Computational Geometry. Ph.,D. Dissertation. University of Illinois Urbana-Champaign, USA."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2962152"},{"key":"e_1_2_2_46_1","volume-title":"SIGMOD Conference. ACM, 553--564","author":"Yakout Mohamed","unstructured":"Mohamed Yakout, Laure Berti-\u00c9 quille, and Ahmed K. Elmagarmid. 2013. Don't be SCAREd: use SCalable Automatic REpairing with maximal likelihood and bounded changes. In SIGMOD Conference. ACM, 553--564."},{"key":"e_1_2_2_47_1","doi-asserted-by":"crossref","unstructured":"Kenji Yamanishi and Jun'ichi Takeuchi. 2002. A unifying framework for detecting outliers and change points from non-stationary time series data. In KDD. ACM 676--681.","DOI":"10.1145\/775047.775148"},{"key":"e_1_2_2_48_1","doi-asserted-by":"crossref","unstructured":"Jie Yang Alisa Smirnova Dingqi Yang Gianluca Demartini Yuan Lu and Philippe Cudr\u00e9 -Mauroux. 2019. Scalpel-CD: Leveraging Crowdsourcing and Deep Probabilistic Modeling for Debugging Noisy Training Data. In WWW. ACM 2158--2168.","DOI":"10.1145\/3308558.3313599"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3320214"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2915233"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.14778\/3115404.3115410"}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654993","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3654993","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T14:40:53Z","timestamp":1755787253000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654993"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,29]]},"references-count":51,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,5,29]]}},"alternative-id":["10.1145\/3654993"],"URL":"https:\/\/doi.org\/10.1145\/3654993","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,29]]}}}