{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T17:19:14Z","timestamp":1780679954893,"version":"3.54.1"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"11","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2021,7]]},"abstract":"<jats:p>Access to high-quality data repositories and benchmarks have been instrumental in advancing the state of the art in many experimental research domains. While advanced analytics tasks over time series data have been gaining lots of attention, lack of such community resources severely limits scientific progress. In this paper, we present Exathlon, the first comprehensive public benchmark for explainable anomaly detection over high-dimensional time series data. Exathlon has been systematically constructed based on real data traces from repeated executions of large-scale stream processing jobs on an Apache Spark cluster. Some of these executions were intentionally disturbed by introducing instances of six different types of anomalous events (e.g., misbehaving inputs, resource contention, process failures). For each of the anomaly instances, ground truth labels for the root cause interval as well as those for the extended effect interval are provided, supporting the development and evaluation of a wide range of anomaly detection (AD) and explanation discovery (ED) tasks. We demonstrate the practical utility of Exathlon's dataset, evaluation methodology, and end-to-end data science pipeline design through an experimental study with three state-of-the-art AD and ED techniques.<\/jats:p>","DOI":"10.14778\/3476249.3476307","type":"journal-article","created":{"date-parts":[[2021,10,27]],"date-time":"2021-10-27T16:46:23Z","timestamp":1635353183000},"page":"2613-2626","source":"Crossref","is-referenced-by-count":68,"title":["Exathlon"],"prefix":"10.14778","volume":"14","author":[{"given":"Vincent","family":"Jacob","sequence":"first","affiliation":[{"name":"Ecole Polytechnique, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fei","family":"Song","sequence":"additional","affiliation":[{"name":"Ecole Polytechnique, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Arnaud","family":"Stiegler","sequence":"additional","affiliation":[{"name":"Ecole Polytechnique, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bijan","family":"Rad","sequence":"additional","affiliation":[{"name":"Ecole Polytechnique, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanlei","family":"Diao","sequence":"additional","affiliation":[{"name":"Ecole Polytechnique, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nesime","family":"Tatbul","sequence":"additional","affiliation":[{"name":"Intel Labs and MIT"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,10,27]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.5555\/1316689.1316732"},{"key":"e_1_2_1_2_1","volume-title":"The UEA Multivariate Time Series Classification Archive, 2018","author":"Bagnall Anthony J.","year":"1811"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3035928"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3455135"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2016.60"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3399579.3399865"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1002\/for.768"},{"key":"e_1_2_1_8_1","volume-title":"Automated Anomaly Detection in Large Sequences. In IEEE 36th International Conference on Data Engineering (ICDE). 1834--1837","author":"Boniol Paul","year":"2020"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.14778\/3407790.3407792"},{"key":"e_1_2_1_10_1","volume-title":"Collective Anomaly Detection Based on Long Short-Term Memory Recurrent Neural Networks. In International Conference on Future Data and Security Engineering (FDSE)","volume":"10018","author":"Bontemps Lo\u00efc","year":"2016"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/342009.335388"},{"key":"e_1_2_1_12_1","volume-title":"Deep Learning for Anomaly Detection: A Survey. CoRR abs\/1901.03407","author":"Chalapathy Raghavendra","year":"2019"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1541880.1541882"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3352020.3352024"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807128.1807152"},{"key":"e_1_2_1_16_1","unstructured":"The Standard Performance Evaluation Corporation. [n.d.]. SPEC Benchmarks. https:\/\/www.spec.org\/ Accessed: 2021-07-27.  The Standard Performance Evaluation Corporation. [n.d.]. SPEC Benchmarks. https:\/\/www.spec.org\/ Accessed: 2021-07-27."},{"key":"e_1_2_1_17_1","unstructured":"The Transaction Processing Council. [n.d.]. TPC Benchmarks. http:\/\/www.tpc.org\/ Accessed: 2021-07-27.  The Transaction Processing Council. [n.d.]. TPC Benchmarks. http:\/\/www.tpc.org\/ Accessed: 2021-07-27."},{"key":"e_1_2_1_18_1","volume-title":"Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn J. Keogh.","author":"Dau Hoang Anh","year":"2018"},{"key":"e_1_2_1_19_1","volume-title":"ImageNet: A Large-Scale Hierarchical Image Database. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 248--255","author":"Deng Jia","year":"2009"},{"key":"e_1_2_1_20_1","volume-title":"The UCI Machine Learning Repository","author":"Dua Dheeru"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.14778\/2735461.2735467"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2500853.2500858"},{"key":"e_1_2_1_23_1","unstructured":"FICO. 2018. Explainable Machine Learning Challenge. https:\/\/community.fico.com\/s\/explainable-machine-learning-challenge Accessed: 2021-07-27.  FICO. 2018. Explainable Machine Learning Challenge. https:\/\/community.fico.com\/s\/explainable-machine-learning-challenge Accessed: 2021-07-27."},{"key":"e_1_2_1_24_1","volume-title":"International Workshop on Comprehensibility and Explanation in AI and ML.","author":"Frosst Nicholas"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2463712"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.5555\/530588"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/2621979"},{"key":"e_1_2_1_28_1","volume-title":"Reducing the Dimensionality of Data with Neural Networks. Science 313, 5786","author":"Hinton Geoffrey","year":"2006"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476249.3476307"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3314048"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1594977.1592597"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2020.113187"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939874"},{"key":"e_1_2_1_34_1","volume-title":"Evaluating Real-Time Anomaly Detection Algorithms - The Numenta Anomaly Benchmark. In IEEE International Conference on Machine Learning and Applications (ICMLA). 38--44","author":"Lavin Alexander","year":"2015"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2389241.2389246"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2133360.2133363"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295230"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.14778\/3389133.3389136"},{"key":"e_1_2_1_39_1","volume-title":"Long Short Term Memory Networks for Anomaly Detection in Time Series. In European Symposium on Artificial Neural Networks (ESANN). 89--94","author":"Malhotra Pankaj","year":"2015"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/219717.219748"},{"key":"e_1_2_1_41_1","unstructured":"Christoph Molnar. 2021. Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. https:\/\/christophm.github.io\/interpretable-ml-book\/ Accessed: 2021-07-27.  Christoph Molnar. 2021. Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. https:\/\/christophm.github.io\/interpretable-ml-book\/ Accessed: 2021-07-27."},{"key":"e_1_2_1_42_1","volume-title":"TPC Technology Conference on Performance Evaluation and Benchmarking (TPCTC). 47--63","author":"Rabl Tilmann","year":"2019"},{"key":"e_1_2_1_43_1","volume-title":"Outlier Detection DataSets","author":"Rayana Shebuti","year":"2021"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_2_1_45_1","volume-title":"Anchors: High-Precision Model-Agnostic Explanations. AAAI Conference.","author":"Ribeiro Marco T\u00falio","year":"2018"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.14778\/2856318.2856329"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-59050-9_12"},{"key":"e_1_2_1_48_1","volume-title":"Demystifying Numenta Anomaly Benchmark. In International Joint Conference on Neural Networks (IJCNN). 1570--1577","author":"Singh Nidhi","year":"2017"},{"key":"e_1_2_1_49_1","unstructured":"Spark-uses [n.d.]. How are Big Companies using Apache Spark. https:\/\/medium.com\/@tao_66792\/how-are-big-companies-using-apache-spark-413743dbbbae Accessed: 2021-07-27.  Spark-uses [n.d.]. How are Big Companies using Apache Spark. https:\/\/medium.com\/@tao_66792\/how-are-big-companies-using-apache-spark-413743dbbbae Accessed: 2021-07-27."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.5555\/3305890.3306024"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.5555\/3326943.3327120"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.14778\/2994509.2994526"},{"key":"e_1_2_1_53_1","volume-title":"GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In International Conference on Learning Representations (ICLR).","author":"Wang Alex"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536354.2536356"},{"key":"e_1_2_1_55_1","volume-title":"Beyond Sparsity: Tree Regularization of Deep Models for Interpretability. AAAI Conference.","author":"Wu Mike","year":"2018"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3185996"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3371425.3371427"},{"key":"e_1_2_1_58_1","volume-title":"EXstream: Explaining Anomalies in Event Stream Monitoring. In International Conference on Extending Database Technology (EDBT). 156--167","author":"Zhang Haopeng","year":"2017"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3476249.3476307","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T10:10:28Z","timestamp":1672222228000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3476249.3476307"}},"subtitle":["a benchmark for explainable anomaly detection over time series"],"short-title":[],"issued":{"date-parts":[[2021,7]]},"references-count":58,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2021,7]]}},"alternative-id":["10.14778\/3476249.3476307"],"URL":"https:\/\/doi.org\/10.14778\/3476249.3476307","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2021,7]]}}}