{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,17]],"date-time":"2026-01-17T21:10:28Z","timestamp":1768684228736,"version":"3.49.0"},"reference-count":47,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2021,2,6]],"date-time":"2021-02-06T00:00:00Z","timestamp":1612569600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Co-Construction with Beijing Municipal Commission of Education of China","award":["B19H100010, B18H100040 and K19H100010"],"award-info":[{"award-number":["B19H100010, B18H100040 and K19H100010"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We address the problem of unsupervised anomaly detection for multivariate data. Traditional machine learning based anomaly detection algorithms rely on specific assumptions of normal patterns and fail to model complex feature interactions and relations. Recently, existing deep learning based methods are promising for extracting representations from complex features. These methods train an auxiliary task, e.g., reconstruction and prediction, on normal samples. They further assume that anomalies fail to perform well on the auxiliary task since they are never trained during the model optimization. However, the assumption does not always hold in practice. Deep models may also perform the auxiliary task well on anomalous samples, leading to the failure detection of anomalies. To effectively detect anomalies for multivariate data, this paper introduces a teacher-student distillation based framework Distillated Teacher-Student Network Ensemble (DTSNE). The paradigm of the teacher-student distillation is able to deal with high-dimensional complex features. In addition, an ensemble of student networks provides a better capability to avoid generalizing the auxiliary task performance on anomalous samples. To validate the effectiveness of our model, we conduct extensive experiments on real-world datasets. Experimental results show superior performance of DTSNE over competing methods. Analysis and discussion towards the behavior of our model are also provided in the experiment section.<\/jats:p>","DOI":"10.3390\/e23020201","type":"journal-article","created":{"date-parts":[[2021,2,7]],"date-time":"2021-02-07T14:07:19Z","timestamp":1612706839000},"page":"201","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Unsupervised Anomaly Detection with Distillated Teacher-Student Network Ensemble"],"prefix":"10.3390","volume":"23","author":[{"given":"Qinfeng","family":"Xiao","sequence":"first","affiliation":[{"name":"School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China"},{"name":"Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing 100044, China"}]},{"given":"Jing","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China"},{"name":"Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing 100044, China"},{"name":"Beijing Laboratory of National Economic Security Early-Warning Engineering, Beijing Jiaotong University, Beijing 100044, China"}]},{"given":"Youfang","family":"Lin","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China"},{"name":"Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing 100044, China"}]},{"given":"Wenbo","family":"Gongsa","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China"},{"name":"Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing 100044, China"}]},{"given":"Ganghui","family":"Hu","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China"},{"name":"Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing 100044, China"}]},{"given":"Menggang","family":"Li","sequence":"additional","affiliation":[{"name":"Beijing Laboratory of National Economic Security Early-Warning Engineering, Beijing Jiaotong University, Beijing 100044, China"},{"name":"National Academy of Economic Security, Beijing Jiaotong University, Beijing 100044, China"}]},{"given":"Fang","family":"Wang","sequence":"additional","affiliation":[{"name":"Beijing Laboratory of National Economic Security Early-Warning Engineering, Beijing Jiaotong University, Beijing 100044, China"},{"name":"National Academy of Economic Security, Beijing Jiaotong University, Beijing 100044, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,2,6]]},"reference":[{"key":"ref_1","unstructured":"Aggarwal, C.C. (2016). Outlier Analysis, Springer Publishing Company, Incorporated. [2nd ed.]."},{"key":"ref_2","unstructured":"Pang, G., Shen, C., Cao, L., and van den Hengel, A. (2020). Deep Learning for Anomaly Detection: A Review. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"113303","DOI":"10.1016\/j.dss.2020.113303","article-title":"Fraud detection: A systematic literature review of graph-based anomaly detection approaches","volume":"133","author":"Pourhabibi","year":"2020","journal-title":"Decis. Support Syst."},{"key":"ref_4","first-page":"440","article-title":"Traffic Anomaly Detection for Data Communication Networks","volume":"Volume 12240","author":"Sun","year":"2020","journal-title":"Proceedings of the 6th International Conference on Artificial Intelligence and Security (ICAIS)"},{"key":"ref_5","unstructured":"Chen, W., Naughton, J.F., and Bernstein, P.A. (2000, January 16\u201318). LOF: Identifying Density-Based Local Outliers. Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, Dallas, TX, USA."},{"key":"ref_6","unstructured":"Chen, W., Naughton, J.F., and Bernstein, P.A. (2000, January 16\u201318). Efficient Algorithms for Mining Outliers from Large Data Sets. Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, Dallas, TX, USA."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Liu, F.T., Ting, K.M., and Zhou, Z. (2008, January 15\u201319). Isolation Forest. Proceedings of the 8th IEEE International Conference on Data Mining (ICDM 2008), Pisa, Italy.","DOI":"10.1109\/ICDM.2008.17"},{"key":"ref_8","unstructured":"Li, Y., Liu, B., and Sarawagi, S. (2008, January 24\u201327). Angle-based outlier detection in high-dimensional data. Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, NV, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Li, Z., Zhao, Y., Botta, N., Ionescu, C., and Hu, X. (2020). COPOD: Copula-Based Outlier Detection. arXiv.","DOI":"10.1109\/ICDM50108.2020.00135"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zenati, H., Romain, M., Foo, C., Lecouat, B., and Chandrasekhar, V. (2018, January 17\u201320). Adversarially Learned Anomaly Detection. Proceedings of the IEEE International Conference on Data Mining (ICDM 2018), Singapore.","DOI":"10.1109\/ICDM.2018.00088"},{"key":"ref_12","unstructured":"Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). Generative Probabilistic Novelty Detection with Adversarial Autoencoders. Advances in Neural Information Processing Systems 31: Proceedings of the Annual Conference on Neural Information Processing Systems 2018 (NeurIPS 2018), Montr\u00e9al, QC, Canada, 3\u20138 December 2018, Curran Associates Inc."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Liu, W., Luo, W., Lian, D., and Gao, S. (2018, January 18\u201322). Future Frame Prediction for Anomaly Detection\u2014A New Baseline. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00684"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Abati, D., Porrello, A., Calderara, S., and Cucchiara, R. (2019, January 16\u201320). Latent Space Autoregression for Novelty Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00057"},{"key":"ref_15","unstructured":"Amsaleg, L., Huet, B., Larson, M.A., Gravier, G., Hung, H., Ngo, C., and Ooi, W.T. (2019, January 21\u201325). AnoPCN: Video Anomaly Detection via Deep Predictive Coding Network. Proceedings of the 27th ACM International Conference on Multimedia (MM 2019), Nice, France."},{"key":"ref_16","unstructured":"Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). Deep Anomaly Detection Using Geometric Transformations. Advances in Neural Information Processing Systems 31: Proceedings of the Annual Conference on Neural Information Processing Systems 2018 (NeurIPS 2018), Montr\u00e9al, QC, Canada, 3\u20138 December 2018, Curran Associates Inc."},{"key":"ref_17","unstructured":"Wallach, H.M., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E.B., and Garnett, R. (2019). Effective End-to-end Unsupervised Outlier Detection via Inlier Priority of Discriminative Network. Advances in Neural Information Processing Systems 32: Proceedings of the Annual Conference on Neural Information Processing Systems 2019 (NeurIPS 2019), Vancouver, BC, Canada, 8\u201314 December 2019, Curran Associates Inc."},{"key":"ref_18","unstructured":"Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., and Weinberger, K.Q. (2014). Generative Adversarial Nets. Advances in Neural Information Processing Systems 27: Proceedings of the Annual Conference on Neural Information Processing Systems 2014, Montreal, QC, Canada, 8\u201313 December 2014, Curran Associates Inc."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1016\/S0167-8655(99)00087-2","article-title":"Support vector domain description","volume":"20","author":"Tax","year":"1999","journal-title":"Pattern Recognit. Lett."},{"key":"ref_20","unstructured":"Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). CatBoost: Unbiased boosting with categorical features. Advances in Neural Information Processing Systems 31: Proceedings of the Annual Conference on Neural Information Processing Systems 2018 (NeurIPS 2018), Montr\u00e9al, QC, Canada, 3\u20138 December 2018, Curran Associates Inc."},{"key":"ref_21","unstructured":"Krishnapuram, B., Shah, M., Smola, A.J., Aggarwal, C.C., Shen, D., and Rastogi, R. (2016, January 13\u201317). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA."},{"key":"ref_22","unstructured":"Teredesai, A., Kumar, V., Li, Y., Rosales, R., Terzi, E., and Karypis, G. (2019, January 4\u20138). Deep Anomaly Detection with Deviation Networks. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD 2019), Anchorage, AK, USA."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Pang, G., van den Hengel, A., and Shen, C. (2019). Weakly-supervised Deep Anomaly Detection with Pairwise Relation Learning. arXiv.","DOI":"10.1145\/3292500.3330871"},{"key":"ref_24","unstructured":"Guo, Y., and Farooq, F. (2018, January 19\u201323). Learning Representations of Ultrahigh-dimensional Data for Random Distance-based Outlier Detection. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD 2018), London, UK."},{"key":"ref_25","unstructured":"Bessiere, C. (2020, January 11\u201317). Unsupervised Representation Learning by Predicting Random Distances. Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI 2020), Yokohama, Japan."},{"key":"ref_26","unstructured":"Kementsietsidis, A., and Salles, M.A.V. (2012, January 1\u20135). HiCS: High Contrast Subspaces for Density-Based Outlier Ranking. Proceedings of the IEEE 28th International Conference on Data Engineering (ICDE 2012), Arlington, VA, USA."},{"key":"ref_27","unstructured":"Grossman, R., Bayardo, R.J., and Bennett, K.P. (2005, January 21\u201324). Feature bagging for outlier detection. Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Chicago, IL, USA."},{"key":"ref_28","unstructured":"Zaki, M.J., Siebes, A., Yu, J.X., Goethals, B., Webb, G.I., and Wu, X. (2012, January 10\u201313). GPU-Accelerated Feature Selection for Outlier Detection Using the Local Kernel Density Ratio. Proceedings of the 12th IEEE International Conference on Data Mining (ICDM 2012), Brussels, Belgium."},{"key":"ref_29","unstructured":"McIlraith, S.A., and Weinberger, K.Q. (2018, January 2\u20137). Sparse Modeling-Based Sequential Ensemble Learning for Effective Outlier Detection in High-Dimensional Numeric Data. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18), the 30th Innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, LA, USA."},{"key":"ref_30","unstructured":"Sierra, C. (2017, January 19\u201325). Learning Homophily Couplings from Non-IID Data for Joint Feature Selection and Noise-Resilient Outlier Detection. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI 2017), Melbourne, Australia."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1007\/s10994-015-5521-0","article-title":"Loda: Lightweight on-line detector of anomalies","volume":"102","year":"2016","journal-title":"Mach. Learn."},{"key":"ref_32","unstructured":"Bergman, L., and Hoshen, Y. (2020, January 26\u201330). Classification-Based Anomaly Detection for General Data. Proceedings of the 8th International Conference on Learning Representations (ICLR 2020), Addis Ababa, Ethiopia."},{"key":"ref_33","unstructured":"Maas, A.L., Hannun, A.Y., and Ng, A.Y. (2013, January 16\u201321). Rectifier nonlinearities improve neural network acoustic models. Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, USA."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Doersch, C., Gupta, A., and Efros, A.A. (2015, January 7\u201313). Unsupervised Visual Representation Learning by Context Prediction. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV 2015), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.167"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1007\/978-3-319-46466-4_5","article-title":"Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles","volume":"Volume 9910","author":"Leibe","year":"2016","journal-title":"Proceedings of the 14th European Conference on Computer Vision (ECCV 2016)"},{"key":"ref_36","unstructured":"Gidaris, S., Singh, P., and Komodakis, N. (May, January 30). Unsupervised Representation Learning by Predicting Image Rotations. Proceedings of the 6th International Conference on Learning Representations (ICLR 2018), Vancouver, BC, Canada."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7\u201313). Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV 2015), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.123"},{"key":"ref_38","unstructured":"Kambhampati, S. (2016, January 9\u201315). Outlier Detection in Complex Categorical Data by Modeling the Feature Value Couplings. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI 2016), New York, NY, USA."},{"key":"ref_39","unstructured":"Bonchi, F., Domingo-Ferrer, J., Baeza-Yates, R., Zhou, Z., and Wu, X. (2016, January 12\u201315). Unsupervised Feature Selection for Outlier Detection by Modelling Hierarchical Value-Feature Couplings. Proceedings of the IEEE 16th International Conference on Data Mining (ICDM 2016), Barcelona, Spain."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"3:1","DOI":"10.1145\/2133360.2133363","article-title":"Isolation-Based Anomaly Detection","volume":"6","author":"Liu","year":"2012","journal-title":"ACM Trans. Knowl. Discov. Data"},{"key":"ref_41","unstructured":"Zong, B., Song, Q., Min, M.R., Cheng, W., Lumezanu, C., Cho, D., and Chen, H. (May, January 30). Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection. Proceedings of the 6th International Conference on Learning Representations (ICLR 2018), Vancouver, BC, Canada."},{"key":"ref_42","first-page":"96:1","article-title":"PyOD: A Python Toolbox for Scalable Outlier Detection","volume":"20","author":"Zhao","year":"2019","journal-title":"J. Mach. Learn. Res."},{"key":"ref_43","unstructured":"Wallach, H.M., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E.B., and Garnett, R. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems 32: Proceedings of the Annual Conference on Neural Information Processing Systems 2019 (NeurIPS 2019), Vancouver, BC, Canada, 8\u201314 December 2019, Curran Associates Inc."},{"key":"ref_44","first-page":"1","article-title":"Statistical Comparisons of Classifiers over Multiple Data Sets","volume":"7","author":"Demsar","year":"2006","journal-title":"J. Mach. Learn. Res."},{"key":"ref_45","unstructured":"Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). Predictive Uncertainty Estimation via Prior Networks. Advances in Neural Information Processing Systems 31: Proceedings of the Annual Conference on Neural Information Processing Systems 2018 (NeurIPS 2018), Montr\u00e9al, QC, Canada, 3\u20138 December 2018, Curran Associates Inc."},{"key":"ref_46","unstructured":"Wallach, H.M., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E.B., and Garnett, R. (2019). Can you trust your model\u2019s uncertainty? Evaluating predictive uncertainty under dataset shift. Advances in Neural Information Processing Systems 32: Proceedings of the Annual Conference on Neural Information Processing Systems 2019(NeurIPS 2019), Vancouver, BC, Canada, 8\u201314 December 2019, Curran Associates Inc."},{"key":"ref_47","unstructured":"Ruff, L., Vandermeulen, R.A., G\u00f6rnitz, N., Binder, A., M\u00fcller, E., M\u00fcller, K., and Kloft, M. (2020, January 26\u201330). Deep Semi-Supervised Anomaly Detection. Proceedings of the 8th International Conference on Learning Representations (ICLR 2020), Addis Ababa, Ethiopia."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/2\/201\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:20:21Z","timestamp":1760160021000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/2\/201"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,6]]},"references-count":47,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["e23020201"],"URL":"https:\/\/doi.org\/10.3390\/e23020201","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,6]]}}}