{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T03:40:35Z","timestamp":1771299635972,"version":"3.50.1"},"reference-count":75,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,5,26]],"date-time":"2023-05-26T00:00:00Z","timestamp":1685059200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Department of Education","award":["P200A180088"],"award-info":[{"award-number":["P200A180088"]}]},{"name":"NSF","award":["IIS- 1910880CSSI-2103832 CSSI- 2103799 CNS-1852498"],"award-info":[{"award-number":["IIS- 1910880CSSI-2103832 CSSI- 2103799 CNS-1852498"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2023,5,26]]},"abstract":"<jats:p>Outlier detection is critical in real world. Due to the existence of many outlier detection techniques which often return different results for the same data set, the users have to address the problem of determining which among these techniques is the best suited for their task and tune its parameters. This is particularly challenging in the unsupervised setting, where no labels are available for cross-validation needed for such method and parameter optimization. In this work, we propose AutoOD which uses the existing unsupervised detection techniques to automatically produce high quality outliers without any human tuning. AutoOD's fundamentally new strategy unifies the merits of unsupervised outlier detection and supervised classification within one integrated solution. It automatically tests a diverse set of unsupervised outlier detectors on a target data set, extracts useful signals from their combined detection results to reliably capture key differences between outliers and inliers. It then uses these signals to produce a \"custom outlier classifier\" to classify outliers, with its accuracy comparable to supervised outlier classification models trained with ground truth labels - without having access to the much needed labels. On a diverse set of benchmark outlier detection datasets, AutoOD consistently outperforms the best unsupervised outlier detector selected from hundreds of detectors. It also outperforms other tuning-free approaches from 12 to 97 points (out of 100) in the F-1 score.<\/jats:p>","DOI":"10.1145\/3588700","type":"journal-article","created":{"date-parts":[[2023,5,30]],"date-time":"2023-05-30T17:42:05Z","timestamp":1685468525000},"page":"1-27","source":"Crossref","is-referenced-by-count":8,"title":["AutoOD: Automatic Outlier Detection"],"prefix":"10.1145","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9909-8607","authenticated-orcid":false,"given":"Lei","family":"Cao","sequence":"first","affiliation":[{"name":"University of Arizona &amp; Massachusetts Institute of Technology, Tucson, AZ, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-5457-0411","authenticated-orcid":false,"given":"Yizhou","family":"Yan","sequence":"additional","affiliation":[{"name":"Worcester Polytechnic Institute, Worcester, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0294-8706","authenticated-orcid":false,"given":"Yu","family":"Wang","sequence":"additional","affiliation":[{"name":"University of California, San Diego, La Jolla, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7470-3265","authenticated-orcid":false,"given":"Samuel","family":"Madden","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5375-9254","authenticated-orcid":false,"given":"Elke A.","family":"Rundensteiner","sequence":"additional","affiliation":[{"name":"Worcester Polytechnic Institute, Worcester, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,5,30]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"2017. Forrester Prediction. https:\/\/go.forrester.com."},{"key":"e_1_2_2_2_1","unstructured":"2022. AutoOD: Automatic Outlier Detection. https:\/\/drive.google.com\/file\/d\/1ZQMoySvarne4hP_UfHcs6aB3dhffPyY0\/view?usp=sharing."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00057"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-47578-3"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2830544.2830549"},{"key":"e_1_2_2_6_1","volume-title":"Outlier ensembles: An introduction","author":"Aggarwal Charu C","unstructured":"Charu C Aggarwal and Saket Sathe. 2017. Outlier ensembles: An introduction. Springer."},{"key":"e_1_2_2_7_1","volume-title":"Asian conference on computer vision. Springer, 622--637","author":"Akcay Samet","year":"2018","unstructured":"Samet Akcay, Amir Atapour-Abarghouei, and Toby P Breckon. 2018. Ganomaly: Semi-supervised anomaly detection via adversarial training. In Asian conference on computer vision. Springer, 622--637."},{"key":"e_1_2_2_8_1","volume-title":"Identity Fraud: Securing the Connected Life.","author":"Al Pascual Sarah Miller","year":"2017","unstructured":"Sarah Miller Al Pascual, Kyle Marchini. 2017. Identity Fraud: Securing the Connected Life. (2017)."},{"key":"e_1_2_2_9_1","first-page":"21","article-title":"Detecting anomalous data using auto-encoders","volume":"6","author":"Andrews Jerone TA","year":"2016","unstructured":"Jerone TA Andrews, Edward J Morton, and Lewis D Griffin. 2016. Detecting anomalous data using auto-encoders. International Journal of Machine Learning and Computing 6, 1 (2016), 21.","journal-title":"International Journal of Machine Learning and Computing"},{"key":"e_1_2_2_10_1","doi-asserted-by":"crossref","unstructured":"Fabrizio Angiulli and Clara Pizzuti. 2002. Fast Outlier Detection in High Dimensional Spaces. In PKDD. 15--26.","DOI":"10.1007\/3-540-45681-3_2"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3035928"},{"key":"e_1_2_2_12_1","unstructured":"Vic Barnett Toby Lewis et al. 1994. Outliers in statistical data. Vol. 3. Wiley New York."},{"key":"e_1_2_2_13_1","volume-title":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 206--222","author":"Beggel Laura","year":"2019","unstructured":"Laura Beggel, Michael Pfeiffer, and Bernd Bischl. 2019. Robust anomaly detection in images using adversarial autoencoders. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 206--222."},{"key":"e_1_2_2_14_1","first-page":"281","article-title":"Random search for hyper-parameter optimization","author":"Bergstra James","year":"2012","unstructured":"James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. Journal of machine learning research 13, Feb (2012), 281--305.","journal-title":"Journal of machine learning research 13"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209889.3209891"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/342009.335388"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-015-0444-8"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783387"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2017.143"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389772"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-71249-9_3"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1541880.1541882"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611974973.11"},{"key":"e_1_2_2_24_1","volume-title":"Robust Variational Autoencoders for Outlier Detection and Repair of Mixed-Type Data. In The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26--28","author":"Eduardo Sim\u00e3o","year":"2020","unstructured":"Sim\u00e3o Eduardo, Alfredo Naz\u00e1bal, Christopher K. I. Williams, and Charles Sutton. 2020. Robust Variational Autoencoders for Outlier Detection and Repair of Mixed-Type Data. In The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26--28 August 2020, Online [Palermo, Sicily, Italy]. 4056--4066."},{"key":"e_1_2_2_25_1","unstructured":"Matthias Feurer Aaron Klein Katharina Eggensperger Jost Springenberg Manuel Blum and Frank Hutter. 2015. Efficient and robust automated machine learning. In Advances in neural information processing systems. 2962--2970."},{"key":"e_1_2_2_26_1","volume-title":"SDM, May 2--4","author":"Gao Jing","year":"2013","unstructured":"Jing Gao, Jiawei Han, Jialu Liu, and Chi Wang. 2013. Multi-View Clustering via Joint Nonnegative Matrix Factorization. In SDM, May 2--4, 2013. Austin, Texas, USA. 252--260."},{"key":"e_1_2_2_27_1","unstructured":"Izhak Golan and Ran El-Yaniv. 2018. Deep anomaly detection using geometric transformations. In NeurIPS. 9758--9769."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098043"},{"key":"e_1_2_2_29_1","volume-title":"Deep Learning","author":"Goodfellow Ian","unstructured":"Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press."},{"key":"e_1_2_2_30_1","volume-title":"Friedman","author":"Hastie Trevor","year":"2009","unstructured":"Trevor Hastie, Robert Tibshirani, and Jerome H. Friedman. 2009. The elements of statistical learning: data mining, inference, and prediction, 2nd Edition. Springer.","edition":"2"},{"key":"e_1_2_2_31_1","volume-title":"Identification of outliers","author":"Hawkins Douglas M","unstructured":"Douglas M Hawkins. 1980. Identification of outliers. Vol. 11. Springer."},{"key":"e_1_2_2_32_1","volume-title":"AutoML: A Survey of the State-of-the-Art. arXiv preprint arXiv:1908.00709","author":"He Xin","year":"2019","unstructured":"Xin He, Kaiyong Zhao, and Xiaowen Chu. 2019. AutoML: A Survey of the State-of-the-Art. arXiv preprint arXiv:1908.00709 (2019)."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-8655(03)00003-5"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.14778\/3554821.3554880"},{"key":"e_1_2_2_35_1","volume-title":"Logistic regression","author":"Kleinbaum David G","unstructured":"David G Kleinbaum, K Dietz, M Gail, Mitchel Klein, and Mitchell Klein. 2002. Logistic regression. Springer."},{"key":"e_1_2_2_36_1","volume-title":"Ng","author":"Knorr Edwin M.","year":"1998","unstructured":"Edwin M. Knorr and Raymond T. Ng. 1998. Algorithms for Mining Distance-Based Outliers in Large Datasets. In VLDB. 392--403."},{"key":"e_1_2_2_37_1","first-page":"211","article-title":"Finding intensional knowledge of distance-based outliers","volume":"99","author":"Knorr Edwin M","year":"1999","unstructured":"Edwin M Knorr and Raymond T Ng. 1999. Finding intensional knowledge of distance-based outliers. In VLDB, Vol. 99. 211--222.","journal-title":"VLDB"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.5555\/3122009.3122034"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1401890.1401946"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1081870.1081891"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.5555\/3122009.3242042"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3187009.3177737"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3366424.3383530"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2008.17"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/s13721-016-0125-6"},{"key":"e_1_2_2_46_1","volume-title":"A Large-scale Study on Unsupervised Outlier Model Selection: Do Internal Strategies Suffice? ArXiv abs\/2104.01422","author":"Ma Martin Q.","year":"2021","unstructured":"Martin Q. Ma, Yue Zhao, Xiaorong Zhang, and Leman Akoglu. 2021. A Large-scale Study on Unsupervised Outlier Model Selection: Do Internal Strategies Suffice? ArXiv abs\/2104.01422 (2021)."},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00453-012-9721-8"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2003.1260802"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_2_2_50_1","first-page":"2825","article-title":"Scikit-learn: Machine learning in Python","author":"Pedregosa Fabian","year":"2011","unstructured":"Fabian Pedregosa, Ga\u00ebl Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al . 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12, Oct (2011), 2825--2830.","journal-title":"Journal of Machine Learning Research 12"},{"key":"e_1_2_2_51_1","volume-title":"Ocgan: One-class novelty detection using gans with constrained latent representations. In CVPR. 2898--2906.","author":"Perera Pramuditha","year":"2019","unstructured":"Pramuditha Perera, Ramesh Nallapati, and Bing Xiang. 2019. Ocgan: One-class novelty detection using gans with constrained latent representations. In CVPR. 2898--2906."},{"key":"e_1_2_2_52_1","unstructured":"Tom\u00e1? Pevn"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-015-5521-0"},{"key":"e_1_2_2_54_1","doi-asserted-by":"crossref","unstructured":"John Platt et al . 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10 3 (1999) 61--74.","DOI":"10.7551\/mitpress\/1113.003.0008"},{"key":"e_1_2_2_55_1","doi-asserted-by":"crossref","unstructured":"Sridhar Ramaswamy Rajeev Rastogi and Kyuseok Shim. 2000. Efficient Algorithms for Mining Outliers from Large Data Sets. In SIGMOD. 427--438.","DOI":"10.1145\/335191.335437"},{"key":"e_1_2_2_56_1","volume-title":"Snorkel: Rapid training data creation with weak supervision. The VLDB Journal","author":"Ratner Alexander","year":"2019","unstructured":"Alexander Ratner, Stephen H Bach, Henry Ehrenberg, Jason Fries, Sen Wu, and Christopher R\u00e9. 2019. Snorkel: Rapid training data creation with weak supervision. The VLDB Journal (2019), 1--22."},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.14778\/3157794.3157797"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33014763"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-009-9124-7"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/2689746.2689747"},{"key":"e_1_2_2_61_1","doi-asserted-by":"crossref","unstructured":"Erich Schubert Remigius Wojdanowski Arthur Zimek and Hans-Peter Kriegel. 2012. On Evaluation of Outlier Rankings and Outlier Scores. In SDM. 1047--1058.","DOI":"10.1137\/1.9781611972825.90"},{"key":"e_1_2_2_62_1","doi-asserted-by":"crossref","unstructured":"Zeyuan Shang Emanuel Zgraggen Benedetto Buratti Ferdinand Kossmann Philipp Eichmann Yeounoh Chung Carsten Binnig Eli Upfal and Tim Kraska. 2019. Democratizing Data Science through Interactive Curation of ML Pipelines. In SIGMOD. 1171--1188.","DOI":"10.1145\/3299869.3319863"},{"key":"e_1_2_2_63_1","volume-title":"Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"5748","author":"Shen Yanyao","year":"2019","unstructured":"Yanyao Shen and Sujay Sanghavi. 2019. Learning with Bad Training Data via Iterative Trimmed Loss Minimization. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, Long Beach, California, USA, 5739--5748. http:\/\/proceedings.mlr.press\/v97\/shen19e.html"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/2806777.2806945"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487575.2487629"},{"key":"e_1_2_2_66_1","volume-title":"Introduction to the non-asymptotic analysis of random matrices. arXiv preprint arXiv:1011.3027","author":"Vershynin Roman","year":"2010","unstructured":"Roman Vershynin. 2010. Introduction to the non-asymptotic analysis of random matrices. arXiv preprint arXiv:1011.3027 (2010)."},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.177"},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098191"},{"key":"e_1_2_2_69_1","volume-title":"Bruno Lecouat, Gaurav Manek, and Vijay Ramaseshan Chandrasekhar.","author":"Zenati Houssam","year":"2018","unstructured":"Houssam Zenati, Chuan Sheng Foo, Bruno Lecouat, Gaurav Manek, and Vijay Ramaseshan Chandrasekhar. 2018. Efficient gan-based anomaly detection. arXiv preprint arXiv:1802.06222 (2018)."},{"key":"e_1_2_2_70_1","volume-title":"Deep structured energy based models for anomaly detection. arXiv preprint arXiv:1605.07717","author":"Zhai Shuangfei","year":"2016","unstructured":"Shuangfei Zhai, Yu Cheng, Weining Lu, and Zhongfei Zhang. 2016. Deep structured energy based models for anomaly detection. arXiv preprint arXiv:1605.07717 (2016)."},{"key":"e_1_2_2_71_1","volume-title":"ELITE: Robust Deep Anomaly Detection with Meta Gradient. In KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","author":"Zhang Huayi","year":"2021","unstructured":"Huayi Zhang, Lei Cao, Peter VanNostrand, Samuel Madden, and Elke A. Rundensteiner. 2021. ELITE: Robust Deep Anomaly Detection with Meta Gradient. In KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14--18, 2021. 2174--2182."},{"key":"e_1_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611975673.66"},{"key":"e_1_2_2_73_1","volume-title":"Wortman Vaughan (Eds.)","volume":"34","author":"Zhao Yue","year":"2021","unstructured":"Yue Zhao, Ryan Rossi, and Leman Akoglu. 2021. Automatic Unsupervised Outlier Model Selection. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (Eds.), Vol. 34. Curran Associates, Inc., 4489--4502."},{"key":"e_1_2_2_74_1","doi-asserted-by":"crossref","unstructured":"Chong Zhou and Randy C Paffenroth. 2017. Anomaly detection with robust deep autoencoders. In SIGKDD. 665--674.","DOI":"10.1145\/3097983.3098052"},{"key":"e_1_2_2_75_1","volume-title":"Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578","author":"Zoph Barret","year":"2016","unstructured":"Barret Zoph and Quoc V Le. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016)."}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3588700","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3588700","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:47:14Z","timestamp":1750178834000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3588700"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,26]]},"references-count":75,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,5,26]]}},"alternative-id":["10.1145\/3588700"],"URL":"https:\/\/doi.org\/10.1145\/3588700","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,26]]}}}