{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T05:17:18Z","timestamp":1755839838519,"version":"3.41.0"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"4","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,12]]},"abstract":"<jats:p>Flink clusters often suffer from hotspot issues where the monitored job delay and CPU usage keep rising and remain high. This necessitates the detection of anomalous time series to pinpoint the hotspot machines. However, the state-of-the-art unsupervised time series anomaly detection (UTAD) methods are ineffective in this scenario. We identify two main reasons for this. First, the hotspot scenario requires us to pay particular attention to Flink-specific anomalies, e.g., slow-rising and high-level anomalies, which the existing methods struggle to address. Second, the state-of-the-art anomaly detection methods often assume that training datasets do not contain anomalies, but the data collected from the running Flink clusters contains noise, which causes these methods to learn anomalous patterns as normal patterns. In this paper, we first conduct experiments to analyze why existing methods fail in the Flink scenario. To tackle these challenges, we propose a cross-contrastive approach to learn the context information for each timestamp to enable Flink-specific anomaly detection. Then, to address noisy anomalies, we incorporate prior knowledge to set an anomaly boundary to prevent the model from learning anomalous patterns. Extensive experiments show that our method not only outperforms existing methods in the Flink scenario but also achieves state-of-the-art results on public benchmark datasets.<\/jats:p>","DOI":"10.14778\/3717755.3717773","type":"journal-article","created":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T15:51:49Z","timestamp":1747756309000},"page":"1159-1168","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Noise Matters: Cross Contrastive Learning for Flink Anomaly Detection"],"prefix":"10.14778","volume":"18","author":[{"given":"Zhihao","family":"Zhuang","sequence":"first","affiliation":[{"name":"East China Normal University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yingying","family":"Zhang","sequence":"additional","affiliation":[{"name":"Alibaba Cloud Computing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kai","family":"Zhao","sequence":"additional","affiliation":[{"name":"Aalborg University, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chenjuan","family":"Guo","sequence":"additional","affiliation":[{"name":"East China Normal University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bin","family":"Yang","sequence":"additional","affiliation":[{"name":"East China Normal University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qingsong","family":"Wen","sequence":"additional","affiliation":[{"name":"Squirrel Ai Learning, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lunting","family":"Fan","sequence":"additional","affiliation":[{"name":"Alibaba Cloud Computing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,5,20]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Ahmed Abdulaal Zhuanghua Liu and Tomer Lancewicki. 2021. Practical approach to asynchronous multivariate time series anomaly detection and localization. In SIGKDD. 2485\u20132494.","DOI":"10.1145\/3447548.3467174"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.7910\/DVN\/EBCFKM"},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Markus M Breunig Hans-Peter Kriegel Raymond T Ng and J\u00f6rg Sander. 2000. LOF: identifying density-based local outliers. In SIGMOD. 93\u2013104.","DOI":"10.1145\/342009.335388"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.14778\/3681954.3681957"},{"key":"e_1_2_1_5_1","first-page":"2","volume-title":"Proc. ACM Manag. Data 1","author":"Campos David","year":"2023","unstructured":"David Campos, Miao Zhang, Bin Yang, Tung Kieu, Chenjuan Guo, and Christian S. Jensen. 2023. LightTS: Lightweight Time Series Classification with Adaptive Ensemble Distillation. Proc. ACM Manag. Data 1, 2 (2023), 171:1\u2013171:27."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2022\/394"},{"key":"e_1_2_1_7_1","volume-title":"Active bias: Training more accurate neural networks by emphasizing high variance samples. Advances in Neural Information Processing Systems 30","author":"Chang Haw-Shiuan","year":"2017","unstructured":"Haw-Shiuan Chang, Erik Learned-Miller, and Andrew McCallum. 2017. Active bias: Training more accurate neural networks by emphasizing high variance samples. Advances in Neural Information Processing Systems 30 (2017)."},{"key":"e_1_2_1_8_1","volume-title":"Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecasting. In ICLR.","author":"Chen Peng","year":"2024","unstructured":"Peng Chen, Yingying Zhang, Yunyao Cheng, Yang Shu, Yihang Wang, Qingsong Wen, Bin Yang, and Chenjuan Guo. 2024. Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecasting. In ICLR."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.14778\/3636218.3636231"},{"key":"e_1_2_1_10_1","volume-title":"DACAD: Domain Adaptation Contrastive Learning for Anomaly Detection in Multivariate Time Series. arXiv preprint arXiv:2404.11269","author":"Darban Zahra Zamanzadeh","year":"2024","unstructured":"Zahra Zamanzadeh Darban, Geoffrey I Webb, and Mahsa Salehi. 2024. DACAD: Domain Adaptation Contrastive Learning for Anomaly Detection in Multivariate Time Series. arXiv preprint arXiv:2404.11269 (2024)."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3442381.3449903"},{"key":"e_1_2_1_12_1","volume-title":"Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems 31","author":"Han Bo","year":"2018","unstructured":"Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. 2018. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems 31 (2018)."},{"key":"e_1_2_1_13_1","doi-asserted-by":"crossref","unstructured":"Kyle Hundman Valentino Constantinou Christopher Laporte Ian Colwell and Tom Soderstrom. 2018. Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In SIGKDD. 387\u2013395.","DOI":"10.1145\/3219819.3219845"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-13-1642-5_33"},{"key":"e_1_2_1_15_1","volume-title":"K-Means-based isolation forest. Knowledge-based systems 195","author":"Karczmarek Pawe\u0142","year":"2020","unstructured":"Pawe\u0142 Karczmarek, Adam Kiersztyn, Witold Pedrycz, and Ebru Al. 2020. K-Means-based isolation forest. Knowledge-based systems 195 (2020), 105659."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE53745.2022.00273"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, J. Vanschoren and S. Yeung (Eds.)","volume":"1","author":"Lai Kwei-Herng","year":"2021","unstructured":"Kwei-Herng Lai, Daochen Zha, Junjie Xu, Yue Zhao, Guanchu Wang, and Xia Hu. 2021. Revisiting Time Series Outlier Detection: Definitions and Benchmarks. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, J. Vanschoren and S. Yeung (Eds.), Vol. 1. Curran. https:\/\/datasets-benchmarks-proceedings.neurips.cc\/paper_files\/paper\/2021\/file\/ec5decca5ed3d6b8079e2e7e7bacc9f2-Paper-round1.pdf"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2008.17"},{"key":"e_1_2_1_19_1","volume-title":"Advances in neural information processing systems 30","author":"Malach Eran","year":"2017","unstructured":"Eran Malach and Shai Shalev-Shwartz. 2017. Decoupling\" when to update\" from\" how to update\". Advances in neural information processing systems 30 (2017)."},{"key":"e_1_2_1_20_1","volume-title":"Mathur and Nils Ole Tippenhauer","author":"Aditya","year":"2016","unstructured":"Aditya P. Mathur and Nils Ole Tippenhauer. 2016. SWaT: a water treatment testbed for research and training on ICS security. In CySWater. 31\u201336."},{"key":"e_1_2_1_21_1","unstructured":"Steffen Moritz Frederik Rehbach and Thomas Bartz-Beielstein. 2020. GECCO Industrial Challenge 2019 Dataset: A water quality dataset for the 'Internet of Things: Online Event Detection for Drinking Water Quality Control' competition at the Genetic and Evolutionary Computation Conference 2019 Prague Czech Republic."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.3390\/s20133738"},{"key":"e_1_2_1_23_1","volume-title":"Poul J\u00f8 rgen Jennum, and Christian Igel","author":"Perslev Mathias","year":"2019","unstructured":"Mathias Perslev, Michael Jensen, Sune Darkner, Poul J\u00f8 rgen Jennum, and Christian Igel. 2019. U-Time: A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Staging. In Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alch\u00e9-Buc, E. Fox, and R. Garnett (Eds.), Vol. 32. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2019\/file\/57bafb2c2dfeefba931bb03a835b1fa9-Paper.pdf"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.14778\/3665844.3665863"},{"key":"e_1_2_1_25_1","first-page":"4393","article-title":"Deep One-Class Classification","volume":"80","author":"Ruff Lukas","year":"2018","unstructured":"Lukas Ruff, Robert A. Vandermeulen, Nico G\u00f6rnitz, Lucas Deecke, Shoaib A. Siddiqui, Alexander Binder, Emmanuel M\u00fcller, and Marius Kloft. 2018. Deep One-Class Classification. In ICML, Vol. 80. 4393\u20134402.","journal-title":"ICML"},{"key":"e_1_2_1_26_1","volume-title":"Estimating the support of a high-dimensional distribution. Neural computation 13, 7","author":"Sch\u00f6lkopf Bernhard","year":"2001","unstructured":"Bernhard Sch\u00f6lkopf, John C Platt, John Shawe-Taylor, Alex J Smola, and Robert C Williamson. 2001. Estimating the support of a high-dimensional distribution. Neural computation 13, 7 (2001), 1443\u20131471."},{"key":"e_1_2_1_27_1","unstructured":"Pavel Senin Jessica Lin Xing Wang Tim Oates Sunil Gandhi Arnold P Boedihardjo Crystal Chen and Susan Frankenstein. 2015. Time series anomaly discovery with grammar-based compression.. In Edbt. 481\u2013492."},{"key":"e_1_2_1_28_1","first-page":"13016","article-title":"Timeseries anomaly detection using temporal hierarchical one-class network","volume":"33","author":"Shen Lifeng","year":"2020","unstructured":"Lifeng Shen, Zhuocong Li, and James Kwok. 2020. Timeseries anomaly detection using temporal hierarchical one-class network. Advances in Neural Information Processing Systems 33 (2020), 13016\u201313026.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2022.3152527"},{"key":"e_1_2_1_30_1","volume-title":"RoFormer: Enhanced Transformer with Rotary Position Embedding. CoRR abs\/2104.09864","author":"Su Jianlin","year":"2021","unstructured":"Jianlin Su, Yu Lu, Shengfeng Pan, Bo Wen, and Yunfeng Liu. 2021. RoFormer: Enhanced Transformer with Rotary Position Embedding. CoRR abs\/2104.09864 (2021). arXiv:2104.09864 https:\/\/arxiv.org\/abs\/2104.09864"},{"key":"e_1_2_1_31_1","doi-asserted-by":"crossref","unstructured":"Ya Su Youjian Zhao Chenhao Niu Rong Liu Wei Sun and Dan Pei. 2019. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In SIGKDD. 2828\u20132837.","DOI":"10.1145\/3292500.3330672"},{"key":"e_1_2_1_32_1","doi-asserted-by":"crossref","unstructured":"Ya Su Youjian Zhao Chenhao Niu Rong Liu Wei Sun and Dan Pei. 2019. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In SIGKDD. 2828\u20132837.","DOI":"10.1145\/3292500.3330672"},{"key":"e_1_2_1_33_1","volume-title":"Ada Wai-Chee Fu, and David W Cheung","author":"Tang Jian","year":"2002","unstructured":"Jian Tang, Zhixiang Chen, Ada Wai-Chee Fu, and David W Cheung. 2002. Enhancing effectiveness of outlier detections for low density patterns. In PAKDD. Springer, 535\u2013548."},{"key":"e_1_2_1_34_1","volume-title":"Support vector data description. Machine learning 54","author":"Tax David MJ","year":"2004","unstructured":"David MJ Tax and Robert PW Duin. 2004. Support vector data description. Machine learning 54 (2004), 45\u201366."},{"key":"e_1_2_1_35_1","volume-title":"Attention is all you need. Advances in neural information processing systems 30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i11.17221"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-024-00872-x"},{"key":"e_1_2_1_38_1","first-page":"1","volume-title":"Proc. ACM Manag. Data 1","author":"Wu Xinle","year":"2023","unstructured":"Xinle Wu, Dalin Zhang, Miao Zhang, Chenjuan Guo, Bin Yang, and Christian S. Jensen. 2023. AutoCTS+: Joint Neural Architecture and Hyperparameter Search for Correlated Time Series Forecasting. Proc. ACM Manag. Data 1, 1 (2023), 97:1\u201397:26."},{"key":"e_1_2_1_39_1","volume-title":"Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=LzQQ89U1qm_","author":"Xu Jiehui","year":"2022","unstructured":"Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. 2022. Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=LzQQ89U1qm_"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3371425.3371427"},{"key":"e_1_2_1_41_1","doi-asserted-by":"crossref","unstructured":"Sean Bin Yang Chenjuan Guo Jilin Hu Jian Tang and Bin Yang. 2021. Unsupervised Path Representation Learning with Curriculum Negative Sampling. In IJCAI. 3286\u20133292.","DOI":"10.24963\/ijcai.2021\/452"},{"key":"e_1_2_1_42_1","volume-title":"DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection. In SIGKDD (KDD '23)","author":"Yang Yiyuan","year":"2023","unstructured":"Yiyuan Yang, Chaoli Zhang, Tian Zhou, Qingsong Wen, and Liang Sun. 2023. DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection. In SIGKDD (KDD '23). ACM."},{"key":"e_1_2_1_43_1","volume-title":"Dual t: Reducing estimation error for transition matrix in label-noise learning. Advances in neural information processing systems 33","author":"Yao Yu","year":"2020","unstructured":"Yu Yao, Tongliang Liu, Bo Han, Mingming Gong, Jiankang Deng, Gang Niu, and Masashi Sugiyama. 2020. Dual t: Reducing estimation error for transition matrix in label-noise learning. Advances in neural information processing systems 33 (2020), 7260\u20137271."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00515"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2016.0179"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.14778\/3636218.3636230"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/616"},{"key":"e_1_2_1_48_1","unstructured":"Zhihao Zhuang Yingying Zhang Kai Zhao Chenjuan Guo Bin Yang Qingsong Wen and Lunting Fan. 2024. Appendix. https:\/\/anonymous.4open.science\/r\/ContraAD-265F\/VLDB_Anomaly_detection_appendix.pdf"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3717755.3717773","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T16:16:56Z","timestamp":1747757816000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3717755.3717773"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12]]},"references-count":48,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,12]]}},"alternative-id":["10.14778\/3717755.3717773"],"URL":"https:\/\/doi.org\/10.14778\/3717755.3717773","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,12]]},"assertion":[{"value":"2025-05-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}