{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T20:27:31Z","timestamp":1760300851066,"version":"3.44.0"},"reference-count":18,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:p>Anomaly detection has been extensively implemented in industry. The reality is that an application may have numerous scenarios where anomalies need to be monitored. However, the complete process of anomaly detection will take much time, including data acquisition, data processing, model training, and model deployment. In particular, some simple scenarios do not require building complex anomaly detection models. This results in a waste of resources. To solve these problems, we build an anomaly detection pipeline(ADOps) to modularize each step. For simple anomaly detection scenarios, no programming is required and new anomaly detection tasks can be created by simply modifying the configuration file. In addition, it can also improve the development efficiency of complex anomaly detection models. We show how users create anomaly detection tasks on the anomaly detection pipeline and how engineers use it to develop anomaly detection models.<\/jats:p>","DOI":"10.14778\/3611540.3611618","type":"journal-article","created":{"date-parts":[[2023,9,15]],"date-time":"2023-09-15T11:32:37Z","timestamp":1694777557000},"page":"4050-4053","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["ADOps: An Anomaly Detection Pipeline in Structured Logs"],"prefix":"10.14778","volume":"16","author":[{"given":"Xintong","family":"Song","sequence":"first","affiliation":[{"name":"NetEase Fuxi AI Lab, Hangzhou, China"}]},{"given":"Yusen","family":"Zhu","sequence":"additional","affiliation":[{"name":"NetEase Fuxi AI Lab, Hangzhou, China"}]},{"given":"Jianfei","family":"Wu","sequence":"additional","affiliation":[{"name":"NetEase Fuxi AI Lab, Hangzhou, China"}]},{"given":"Bai","family":"Liu","sequence":"additional","affiliation":[{"name":"NetEase Fuxi AI Lab, Hangzhou, China"}]},{"given":"Hongkang","family":"Wei","sequence":"additional","affiliation":[{"name":"NetEase Fuxi AI Lab, Hangzhou, China"}]}],"member":"320","published-online":{"date-parts":[[2023,8]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Variational autoencoder based anomaly detection using reconstruction probability. Special lecture on IE 2, 1","author":"An Jinwon","year":"2015","unstructured":"Jinwon An and Sungzoon Cho. 2015. Variational autoencoder based anomaly detection using reconstruction probability. Special lecture on IE 2, 1 (2015), 1--18."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3314045"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2017.143"},{"key":"e_1_2_1_4_1","volume-title":"Ehsan Zare Borzeshi, and Massimo Piccardi","author":"Chalapathy Raghavendra","year":"2016","unstructured":"Raghavendra Chalapathy, Ehsan Zare Borzeshi, and Massimo Piccardi. 2016. An investigation of recurrent neural architectures for drug name recognition. arXiv preprint arXiv:1609.07585 (2016)."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1541880.1541882"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.14778\/3446095.3446102"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.14778\/3554821.3554880"},{"key":"e_1_2_1_9_1","volume-title":"Effect of outliers and nonhealthy individuals on reference interval estimation. Clinical chemistry 47, 12","author":"Horn Paul S","year":"2001","unstructured":"Paul S Horn, Lan Feng, Yanmei Li, and Amadeo J Pesce. 2001. Effect of outliers and nonhealthy individuals on reference interval estimation. Clinical chemistry 47, 12 (2001), 2137--2145."},{"key":"e_1_2_1_10_1","unstructured":"Thomas Kluyver Benjamin Ragan-Kelley Fernando P\u00e9rez Brian E Granger Matthias Bussonnier Jonathan Frederic Kyle Kelley Jessica B Hamrick Jason Grout Sylvain Corlay et al. 2016. Jupyter Notebooks-a publishing format for reproducible computational workflows. Vol. 2016."},{"key":"e_1_2_1_11_1","volume-title":"Impala: A Modern, Open-Source SQL Engine for Hadoop.. In Cidr","author":"Kornacker Marcel","year":"2015","unstructured":"Marcel Kornacker, Alexander Behm, Victor Bittorf, Taras Bobrovytsky, Casey Ching, Alan Choi, Justin Erickson, Martin Grund, Daniel Hecht, Matthew Jacobs, et al. 2015. Impala: A Modern, Open-Source SQL Engine for Hadoop.. In Cidr, Vol. 1. Asilomar, CA, 9."},{"key":"e_1_2_1_12_1","volume-title":"Fifth international workshop on intelligent data analysis in medicine and pharmacology","volume":"1","author":"Laurikkala Jorma","year":"2000","unstructured":"Jorma Laurikkala, Martti Juhola, Erna Kentala, N Lavrac, S Miksch, and B Kavsek. 2000. Informal identification of outliers in medical data. In Fifth international workshop on intelligent data analysis in medicine and pharmacology, Vol. 1. 20--24."},{"volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems","author":"Scott","key":"e_1_2_1_13_1","unstructured":"Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS'17). Curran Associates Inc., Red Hook, NY, USA, 4768--4777."},{"key":"e_1_2_1_14_1","first-page":"4739","article-title":"LogAnomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs","volume":"19","author":"Meng Weibin","year":"2019","unstructured":"Weibin Meng, Ying Liu, Yichen Zhu, Shenglin Zhang, Dan Pei, Yuqing Liu, Yihao Chen, Ruizhi Zhang, Shimin Tao, Pei Sun, et al. 2019. LogAnomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs.. In IJCAI, Vol. 19. 4739--4745.","journal-title":"IJCAI"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.25080\/Majora-7b98e3ed-013"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330687"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2021.106878"},{"key":"e_1_2_1_18_1","first-page":"10","article-title":"Spark: Cluster computing with working sets","volume":"10","author":"Zaharia Matei","year":"2010","unstructured":"Matei Zaharia, Mosharaf Chowdhury, Michael J Franklin, Scott Shenker, Ion Stoica, et al. 2010. Spark: Cluster computing with working sets. HotCloud 10, 10--10 (2010), 95.","journal-title":"HotCloud"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3611540.3611618","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,10]],"date-time":"2025-09-10T22:34:42Z","timestamp":1757543682000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3611540.3611618"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8]]},"references-count":18,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2023,8]]}},"alternative-id":["10.14778\/3611540.3611618"],"URL":"https:\/\/doi.org\/10.14778\/3611540.3611618","relation":{},"ISSN":["2150-8097"],"issn-type":[{"type":"print","value":"2150-8097"}],"subject":[],"published":{"date-parts":[[2023,8]]},"assertion":[{"value":"2023-08-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}