{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T05:20:06Z","timestamp":1773724806524,"version":"3.50.1"},"reference-count":58,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T00:00:00Z","timestamp":1672185600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science and Higher Education of the Russian Federation","award":["075-15-2022-311"],"award-info":[{"award-number":["075-15-2022-311"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>A new modification of the isolation forest called the attention-based isolation forest (ABIForest) is proposed for solving the anomaly detection problem. It incorporates an attention mechanism in the form of Nadaraya\u2013Watson regression into the isolation forest to improve the solution of the anomaly detection problem. The main idea underlying the modification is the assignment of attention weights to each path of trees with learnable parameters depending on the instances and trees themselves. Huber\u2019s contamination model is proposed to be used to define the attention weights and their parameters. As a result, the attention weights are linearly dependent on learnable attention parameters that are trained by solving a standard linear or quadratic optimization problem. ABIForest can be viewed as the first modification of the isolation forest to incorporate an attention mechanism in a simple way without applying gradient-based algorithms. Numerical experiments with synthetic and real datasets illustrate that the results of ABIForest outperform those of other methods. The code of the proposed algorithms has been made available.<\/jats:p>","DOI":"10.3390\/a16010019","type":"journal-article","created":{"date-parts":[[2022,12,29]],"date-time":"2022-12-29T02:52:21Z","timestamp":1672282341000},"page":"19","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Improved Anomaly Detection by Using the Attention-Based Isolation Forest"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5637-1420","authenticated-orcid":false,"given":"Lev","family":"Utkin","sequence":"first","affiliation":[{"name":"Higher School of Artificial Intelligence, Peter the Great St. Petersburg Polytechnic University, Polytechnicheskaya, 29, 195251 St. Petersburg, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1562-2676","authenticated-orcid":false,"given":"Andrey","family":"Ageev","sequence":"additional","affiliation":[{"name":"Higher School of Artificial Intelligence, Peter the Great St. Petersburg Polytechnic University, Polytechnicheskaya, 29, 195251 St. Petersburg, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1542-6480","authenticated-orcid":false,"given":"Andrei","family":"Konstantinov","sequence":"additional","affiliation":[{"name":"Higher School of Artificial Intelligence, Peter the Great St. Petersburg Polytechnic University, Polytechnicheskaya, 29, 195251 St. Petersburg, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3583-7324","authenticated-orcid":false,"given":"Vladimir","family":"Muliukha","sequence":"additional","affiliation":[{"name":"Higher School of Artificial Intelligence, Peter the Great St. Petersburg Polytechnic University, Polytechnicheskaya, 29, 195251 St. Petersburg, Russia"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,28]]},"reference":[{"key":"ref_1","unstructured":"Chalapathy, R., and Chawla, S. (2022). Deep learning for anomaly detection: A survey. arXiv."},{"key":"ref_2","first-page":"1","article-title":"Outlier Detection: Methods, Models, and Classification","volume":"53","author":"Boukerche","year":"2020","journal-title":"ACM Comput. Surv."},{"key":"ref_3","unstructured":"Braei, M., and Wagner, S. (2020). Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Art. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1541880.1541882","article-title":"Anomaly detection: A survey","volume":"41","author":"Chandola","year":"2009","journal-title":"ACM Comput. Surv."},{"key":"ref_5","unstructured":"Farizi, W.A., Hidayah, I., and Rizal, M. (2021, January 23\u201324). Isolation Forest Based Anomaly Detection: A Systematic Literature Review. Proceedings of the 8th International Conference on Information Technology, Computer and Electrical Engineering (ICITACEE), Semarang, Indonesia."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2252","DOI":"10.1109\/TSP.2021.3061298","article-title":"Minimax Robust Detection: Classic Results and Recent Advances","volume":"69","author":"Fauss","year":"2021","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3439950","article-title":"Deep Learning for Anomaly Detection: A Review","volume":"54","author":"Pang","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref_8","unstructured":"Yang, J., Zhou, K., Li, Y., and Liu, Z. (2021). Generalized Out-of-Distribution Detection: A Survey. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Pang, G., Cao, L., and Aggarwal, C. (2021, January 8\u201312). Deep Learning for Anomaly Detection: Challenges, Methods, and Opportunities. Proceedings of the 14th ACM International Conference on Web Search and Data Mining, Virtual.","DOI":"10.1145\/3437963.3441659"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"756","DOI":"10.1109\/JPROC.2021.3052449","article-title":"A Unifying Review of Deep and Shallow Anomaly Detection","volume":"109","author":"Ruff","year":"2021","journal-title":"Proc. IEEE"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"107964","DOI":"10.1109\/ACCESS.2019.2932769","article-title":"Progress in Outlier Detection Techniques: A Survey","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Access."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Aggarwal, C. (2013). An Introduction to Outlier Analysis, Springer. Chapter Outlier Analysis.","DOI":"10.1007\/978-1-4614-6396-2"},{"key":"ref_13","first-page":"395","article-title":"A linear programming approach to novelty detection","volume":"Volume 13","author":"Leen","year":"2001","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1443","DOI":"10.1162\/089976601750264965","article-title":"Estimating the support of a high-dimensional distribution","volume":"13","author":"Scholkopf","year":"2001","journal-title":"Neural Comput."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1023\/B:MACH.0000008084.60811.49","article-title":"Support vector data description","volume":"54","author":"Tax","year":"2004","journal-title":"Mach. Learn."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Liu, F.T., Kai, M.T., and Zhou, Z.H. (2008, January 15\u201319). Isolation forest. Proceedings of the 2008 Eighth IEEE International Conference on Data Mining, Pisa, Italy.","DOI":"10.1109\/ICDM.2008.17"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2133360.2133363","article-title":"Isolation-based anomaly detection","volume":"6","author":"Liu","year":"2012","journal-title":"ACM Trans. Knowl. Discov. Data (TKDD)"},{"key":"ref_18","unstructured":"Chaudhari, S., Mithal, V., Polatkan, G., and Ramanath, R. (2019). An attentive survey of attention models. arXiv."},{"key":"ref_19","unstructured":"Correia, A., and Colombini, E. (2021). Attention, please! A survey of neural attention models in deep learning. arXiv."},{"key":"ref_20","unstructured":"Correia, A., and Colombini, E. (2021). Neural Attention Models in Deep Learning: Survey and Taxonomy. arXiv."},{"key":"ref_21","unstructured":"Lin, T., Wang, Y., Liu, X., and Qiu, X. (2021). A Survey of Transformers. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.neucom.2021.03.091","article-title":"A review on the attention mechanism of deep learning","volume":"452","author":"Niu","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.neunet.2022.07.029","article-title":"Attention-based Random Forest and Contamination Model","volume":"154","author":"Utkin","year":"2022","journal-title":"Neural Netw."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1137\/1109020","article-title":"On estimating regression","volume":"9","author":"Nadaraya","year":"1964","journal-title":"Theory Probab. Its Appl."},{"key":"ref_25","first-page":"359","article-title":"Smooth regression analysis","volume":"26","author":"Watson","year":"1964","journal-title":"Sankhya Indian J. Stat. Ser. A"},{"key":"ref_26","unstructured":"Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Huber, P. (1981). Robust Statistics, Wiley.","DOI":"10.1002\/0471725250"},{"key":"ref_28","unstructured":"Sawant, S., and Singh, S. (2020). Understanding Attention: In Minds and Machines. arXiv."},{"key":"ref_29","unstructured":"Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., and Bengio, Y. (2015, January 6\u201311). Show, attend and tell: Neural image caption generation with visual attention. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Luong, T., Pham, H., and Manning, C. (2015, January 17\u201321). Effective approaches to attention-based neural machine translation. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.","DOI":"10.18653\/v1\/D15-1166"},{"key":"ref_31","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, L., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., and Hovy, E. (2016, January 12\u201317). Hierarchical attention networks for document classification. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA.","DOI":"10.18653\/v1\/N16-1174"},{"key":"ref_33","unstructured":"Liu, F., Huang, X., Chen, Y., and Suykens, J. (2021). Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond. arXiv."},{"key":"ref_34","unstructured":"Utkin, L., and Konstantinov, A. (2022). Attention and Self-Attention in Random Forests. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"851","DOI":"10.15622\/ia.21.5.1","article-title":"Random Survival Forests Incorporated by the Nadaraya-Watson Regression","volume":"21","author":"Utkin","year":"2022","journal-title":"Inform. Autom."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Konstantinov, A., Utkin, L., and Kirpichenko, S. (2022, January 27\u201329). AGBoost: Attention-based Modification of Gradient Boosting Machine. Proceedings of the 31st Conference of Open Innovations Association (FRUCT), Helsinki, Finland.","DOI":"10.23919\/FRUCT54823.2022.9770928"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"106795","DOI":"10.1016\/j.epsr.2020.106795","article-title":"A3d: Attention-based auto-encoder anomaly detector for false data injection attacks","volume":"189","author":"Kundu","year":"2020","journal-title":"Electr. Power Syst. Res."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2930","DOI":"10.1080\/08839514.2022.2094885","article-title":"Anomaly Detection Using Siamese Network with Attention Mechanism for Few-Shot Learning","volume":"36","author":"Takimoto","year":"2022","journal-title":"Appl. Artif. Intell."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"109607","DOI":"10.1016\/j.ymssp.2022.109607","article-title":"Mutual information based anomaly detection of monitoring data with attention mechanism and residual learning","volume":"182","author":"Lei","year":"2023","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Yu, Y., Zha, Z., Jin, B., Wu, G., and Dong, C. (2022, January 7\u201311). Graph-Based Anomaly Detection via Attention Mechanism. Proceedings of the International Conference on Intelligent Computing, Xi\u2019an, China.","DOI":"10.1007\/978-3-031-13870-6_33"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Madan, N., Ristea, N.C., Ionescu, R., Nasrollahi, K., Khan, F., Moeslund, T., and Shah, M. (2022). Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection. arXiv.","DOI":"10.1109\/CVPR52688.2022.01321"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Ristea, N.C., Madan, N., Ionescu, R., Nasrollahi, K., Khan, F., Moeslund, T., and Shah, M. (2022, January 18\u201322). Self-supervised predictive convolutional attentive block for anomaly detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52688.2022.01321"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Huang, C., Xu, Q., Wang, Y., Wang, Y., and Zhang, Y. (2022). Self-Supervised Masking for Unsupervised Anomaly Detection and Localization. arXiv.","DOI":"10.1109\/TMM.2022.3175611"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Zhao, H., Wang, Y., Duan, J., Huang, C., Cao, D., Tong, Y., Xu, B., Bai, J., Tong, J., and Zhang, Q. (2020, January 17\u201320). Multivariate time-series anomaly detection via graph attention network. Proceedings of the 2020 IEEE International Conference on Data Mining (ICDM), Sorrento, Italy.","DOI":"10.1109\/ICDM50108.2020.00093"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"106706","DOI":"10.1016\/j.cmpb.2022.106706","article-title":"Weighted IForest and siamese GRU on small sample anomaly detection in healthcare","volume":"218","author":"Wang","year":"2022","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1479","DOI":"10.1109\/TKDE.2019.2947676","article-title":"Extended Isolation Forest","volume":"33","author":"Hariri","year":"2021","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Buschjager, S., Honysz, P.J., and Morik, K. (2020, January 6\u20139). Generalized Isolation Forest: Some Theory and More Applications Extended Abstract. Proceedings of the IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA), Sydney, Australia.","DOI":"10.1109\/DSAA49011.2020.00120"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.patrec.2021.05.022","article-title":"Generalized isolation forest for anomaly detection","volume":"149","author":"Lesouple","year":"2021","journal-title":"Pattern Recognit."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.knosys.2020.105659","article-title":"K-Means-based isolation forest","volume":"195","author":"Karczmarek","year":"2020","journal-title":"Knowl. Based Syst."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Karczmarek, P., Kiersztyn, A., and Pedrycz, W. (2020, January 19\u201324). Fuzzy Set-Based Isolation Forest. Proceedings of the IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Glasgow, UK.","DOI":"10.1109\/FUZZ48607.2020.9177718"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1016\/j.ins.2021.10.075","article-title":"A probabilistic generalization of isolation forest","volume":"584","author":"Tokovarov","year":"2022","journal-title":"Inf. Sci."},{"key":"ref_52","first-page":"1","article-title":"Similarity-measured isolation forest: Anomaly detection method for machine monitoring data","volume":"70","author":"Li","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1109\/TGRS.2019.2936308","article-title":"Hyperspectral Anomaly Detection with Kernel Isolation Forest","volume":"58","author":"Li","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_54","first-page":"1","article-title":"An Optimized Computational Framework for Isolation Forest","volume":"2018","author":"Liu","year":"2018","journal-title":"Math. Probl. Eng."},{"key":"ref_55","unstructured":"Staerman, G., Mozharovskyi, P., Clemencon, S., and d\u2032Alche Buc, F. (2019, January 17\u201319). Functional Isolation Forest. Proceedings of the Eleventh Asian Conference on Machine Learning, Nagoya, Japan."},{"key":"ref_56","unstructured":"Xu, H., Pang, G., Wang, Y., and Wang, Y. (2022). Deep Isolation Forest for Anomaly Detection. arXiv."},{"key":"ref_57","unstructured":"Zhang, A., Lipton, Z., Li, M., and Smola, A. (2021). Dive into Deep Learning. arXiv."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Walley, P. (1991). Statistical Reasoning with Imprecise Probabilities, Chapman and Hall.","DOI":"10.1007\/978-1-4899-3472-7"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/1\/19\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:54:43Z","timestamp":1760147683000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/1\/19"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,28]]},"references-count":58,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["a16010019"],"URL":"https:\/\/doi.org\/10.3390\/a16010019","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,28]]}}}