{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:52:10Z","timestamp":1760147530857,"version":"build-2065373602"},"reference-count":49,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,2,8]],"date-time":"2023-02-08T00:00:00Z","timestamp":1675814400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Korea University","award":["BRI-01805"],"award-info":[{"award-number":["BRI-01805"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the development of the Internet and communication technologies, the types of services provided by multitier Web systems are becoming more diverse and complex compared to those of the past. Ensuring a continuous availability of business services is crucial for multitier Web system providers, as service performance issues immediately affect customer experience and satisfaction. Large companies attempt to monitor the system performance indicator (SPI) that summarizes the status of multitier Web systems to detect performance anomalies at an early stage. However, the current anomaly detection methods are designed to monitor a single specific SPI. Moreover, the existing approaches consider performance anomaly detection and its root cause analysis separately, thereby aggravating the burden of resolving the performance anomaly. To support the system provider in diagnosing the performance anomaly, we propose an advanced causative metric analysis (ACMA) framework. First, we draw out 191 performance metrics (PMs) closely related to the target SPI. Among these PMs, the ACMA determines 62 vital PMs that have the most influence on the variance of the target SPI using several statistical methods. Then, we implement a performance anomaly detection model to identify the causative metrics (CMs) between the vital PMs using random forest regression. Even if the target SPI changes, our detection model does not require any change in its model structure and can derive closely related PMs of the target SPI. Based on our experiments, wherein we applied the ACMA to the business services in an enterprise system, we observed that the proposed ACMA could correctly detect various performance anomalies and their CMs.<\/jats:p>","DOI":"10.3390\/s23041919","type":"journal-article","created":{"date-parts":[[2023,2,9]],"date-time":"2023-02-09T01:37:07Z","timestamp":1675906627000},"page":"1919","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Multitier Web System Reliability: Identifying Causative Metrics and Analyzing Performance Anomaly Using a Regression Model"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6992-4207","authenticated-orcid":false,"given":"Sundeuk","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Computer Science, Korea University, Seoul 02841, Republic of Korea"},{"name":"Platform Planning Group, Samsung SDS, Seoul 05510, Republic of Korea"}]},{"given":"Jong Seon","family":"Kim","sequence":"additional","affiliation":[{"name":"MSP Development Group, Samsung SDS, Seoul 05510, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4192-4122","authenticated-orcid":false,"given":"Hoh Peter","family":"In","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Korea University, Seoul 02841, Republic of Korea"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2791120","article-title":"Performance anomaly detection and bottleneck identification","volume":"48","author":"Ibidunmoye","year":"2015","journal-title":"ACM Comput. Surv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ren, H., Xu, B., Wang, Y., Yi, C., Huang, C., Kou, X., Xing, T., Yang, M., Tong, J., and Zhang, Q. (2019, January 4\u20138). Time-series anomaly detection service at microsoft. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330680"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Laptev, N., Amizadeh, S., and Flint, I. (2015, January 10\u201313). Generic and scalable framework for automated time-series anomaly detection. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia.","DOI":"10.1145\/2783258.2788611"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Peiris, M., Hill, J.H., Thelin, J., Bykov, S., Kliot, G., and Konig, C. (July, January 27). Pad: Performance anomaly detection in multi-server distributed systems. Proceedings of the 2014 IEEE 7th International Conference on Cloud Computing, Anchorage, AK, USA.","DOI":"10.1109\/CLOUD.2014.107"},{"key":"ref_5","unstructured":"(2022, June 09). Jennifersoft Inc. 2021. Available online: https:\/\/jennifersoft.com\/en\/."},{"key":"ref_6","unstructured":"Samsung SDS (2022, June 16). The Multi-Tier Web System of Samsung Virtual Private Network (SVPN). Available online: https:\/\/www.samsungvpn.com."},{"key":"ref_7","unstructured":"Samsung SDS (2021, June 16). The Multi-Tier Web System of Samsung uReady. Available online: http:\/\/www.samsunguready.com."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wilcox, R.R. (2017). Introduction to Robust Estimation and Hypothesis Testing, Academic Press. [4th ed.].","DOI":"10.1016\/B978-0-12-804733-0.00001-9"},{"key":"ref_9","unstructured":"(2022, December 12). Global Wep Application Market Share. Available online: https:\/\/hostadvice.com\/marketshare\/server\/."},{"key":"ref_10","unstructured":"Gartner (2022, December 10). Market Share Analysis: Server Operating Systems, Worldwide. Available online: https:\/\/www.gartner.com\/en\/documents\/3883365."},{"key":"ref_11","unstructured":"(2022, December 10). Cisco. SNMP Configuration Guide. 2021. Available online: https:\/\/www.cisco.com\/c\/en\/us\/td\/docs\/ios-xml\/ios\/snmp\/configuration\/xe-16\/snmp-xe-16-book\/nm-snmp-cfg-snmp-support.html."},{"key":"ref_12","unstructured":"F5 (2022, December 12). Configuring SNMP. Available online: https:\/\/support.f5.com\/csp\/article\/K52219241."},{"key":"ref_13","unstructured":"Juniper (2022, December 10). Configuring Basic SNMP. Available online: https:\/\/www.juniper.net\/documentation\/us\/en\/software\/junos\/network-mgmt\/topics\/topic-map\/configuring-basic-snmp.html."},{"key":"ref_14","unstructured":"Mauro, D., and Schmidt, K. (2005). Essential SNMP: Help for System and Network Administrators, O\u2019Reilly Media, Inc."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Stallings, W. (1998). SNMP, SNMPv2, SNMPv3, and RMON 1 and 2, Addison-Wesley Longman Publishing Co., Inc.","DOI":"10.1109\/COMST.1998.5340405"},{"key":"ref_16","unstructured":"Gartner (2022, June 09). Market Share: Database Management Systems, Worldwide. Available online: https:\/\/www.gartner.com\/en\/documents\/4001330."},{"key":"ref_17","unstructured":"(2022, June 09). Filebeat Overview. 2021. Available online: https:\/\/www.elastic.co\/guide\/en\/beats\/filebeat\/current\/filebeat-overview.html."},{"key":"ref_18","unstructured":"(2022, June 09). Zabbix LLC. 2021. Available online: https:\/\/www.zabbix.com\/."},{"key":"ref_19","unstructured":"(2022, December 12). Spark Apache. Available online: http:\/\/spark.apache.org\/docs\/latest\/mllib-decision-tree.html."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1007\/s10100-017-0479-6","article-title":"A framework for sensitivity analysis of decision trees","volume":"26","author":"Jakubczyk","year":"2018","journal-title":"Cent. Eur. J. Oper. Res."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Deng, H., Runger, G., and Tuv, E. (2011, January 14\u201317). Bias of importance measures for multi-valued attributes and solutions. Proceedings of the International Conference on Artificial Neural Networks, Espoo, Finland.","DOI":"10.1007\/978-3-642-21738-8_38"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1080\/17517575.2020.1762245","article-title":"Hybrid sampling method for autoregressive classification trees under density-weighted curvature distance","volume":"15","author":"Ye","year":"2021","journal-title":"Enterp. Inf. Syst."},{"key":"ref_23","first-page":"18","article-title":"Classification and regression by randomForest","volume":"2","author":"Liaw","year":"2002","journal-title":"R News"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1770","DOI":"10.1080\/01621459.2015.1036994","article-title":"Reinforcement learning trees","volume":"110","author":"Zhu","year":"2015","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2142","DOI":"10.1109\/TPAMI.2016.2636831","article-title":"Cross-validated variable selection in tree-based methods improves predictive performance","volume":"39","author":"Painsky","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","unstructured":"Vidal, T., and Schiffer, M. (2020, January 26\u201328). Born-again tree ensembles. Proceedings of the International Conference on Machine Learning (PMLR\u201920), Online."},{"key":"ref_27","unstructured":"Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer Science & Business Media."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"120","DOI":"10.20982\/tqmp.13.2.p120","article-title":"How to compare the means of two samples that include paired observations and independent observations: A companion to Derrick, Russ, Toher and White (2017)","volume":"13","author":"Derrick","year":"2017","journal-title":"Quant. Methods Psychol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"732","DOI":"10.1542\/peds.2005-1134","article-title":"William Sealy Gosset and William A. Silverman: Two \u201cstudents\u201d of science","volume":"116","author":"Raju","year":"2005","journal-title":"Pediatrics"},{"key":"ref_30","unstructured":"Walpole, R.E., Myers, R.H., Myers, S.L., and Ye, K. (1993). Probability and Statistics for Engineers and Scientists, Macmillan."},{"key":"ref_31","unstructured":"Bevans, R. (2022, December 12). T-Distribution: What It Is and How to Use It. Available online: https:\/\/www.scribbr.com\/statistics\/t-distribution\/."},{"key":"ref_32","unstructured":"Weisstein, E.W. (2022, December 10). Statistical Correlation. 2020. Available online: https:\/\/mathworld.wolfram.com\/StatisticalCorrelation.html."},{"key":"ref_33","unstructured":"SPSS Tutorials (2022, December 10). Statistical Pearson Correlation. Available online: https:\/\/libguides.library.kent.edu\/SPSS\/PearsonCorr."},{"key":"ref_34","unstructured":"(2022, December 10). Elasticsearch Guide. Available online: https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/rest-apis.html."},{"key":"ref_35","unstructured":"(2022, December 10). Filebeat API. Available online: https:\/\/www.elastic.co\/guide\/en\/beats\/filebeat\/current\/http-endpoint.html."},{"key":"ref_36","unstructured":"(2022, December 10). Jennifer API. Available online: https:\/\/github.com\/jennifersoft\/jennifer-server-view-api-oam."},{"key":"ref_37","unstructured":"(2022, December 10). Zabbix API. Available online: https:\/\/www.zabbix.com\/documentation\/current\/manual\/api."},{"key":"ref_38","unstructured":"(2022, December 10). Logstash. Available online: https:\/\/www.elastic.co\/guide\/en\/logstash\/current\/plugins-inputs-snmp.html."},{"key":"ref_39","unstructured":"(2022, December 10). Kibana Guide. Available online: https:\/\/www.elastic.co\/guide\/en\/kibana\/master\/api.html."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Liu, D., Zhao, Y., Xu, H., Sun, Y., Pei, D., Luo, J., Jing, X., and Feng, M. (2015, January 28\u201330). Opprentice: Towards practical and automatic anomaly detection through machine learning. Proceedings of the 2015 Internet Measurement Conference, Tokyo, Japan.","DOI":"10.1145\/2815675.2815679"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Xu, H., Chen, W., Zhao, N., Li, Z., Bu, J., Li, Z., Liu, Y., Zhao, Y., Pei, D., and Feng, Y. (2018, January 23\u201327). Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications. Proceedings of the 2018 World Wide Web Conference, Lyon, France.","DOI":"10.1145\/3178876.3185996"},{"key":"ref_42","unstructured":"Shipmon, D.T., Gurevitch, J.M., Piselli, P.M., and Edwards, S.T. (2017). Time series anomaly detection; detection of anomalous drops with limited features and sparse examples in noisy highly periodic data. arXiv."},{"key":"ref_43","unstructured":"Vallis, O., Hochenbaum, J., and Kejariwal, A. (2014, January 17\u201318). A Novel for LongTerm Anomaly Detection in the Cloud. Proceedings of the 6th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud\u201914), Philadelphia, PA, USA."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"e1747","DOI":"10.1002\/stvr.1747","article-title":"Automatic monitoring of service reliability for web applications: A simulation-based approach","volume":"30","author":"Kim","year":"2020","journal-title":"Softw. Test. Verif. Reliab."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Kavulya, S.P., Daniels, S., Joshi, K., Hiltunen, M., Gandhi, R., and Narasimhan, P. (2012, January 25\u201328). Draco: Statistical diagnosis of chronic problems in large distributed systems. Proceedings of the IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN 2012), Boston, MA, USA.","DOI":"10.1109\/DSN.2012.6263927"},{"key":"ref_46","unstructured":"Nagaraj, K., Killian, C., and Neville, J. (2012, January 25\u201327). Structured comparative analysis of systems logs to diagnose performance problems. Proceedings of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201912), San Jose, CA, USA."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Roy, S., K\u00f6nig, A.C., Dvorkin, I., and Kumar, M. (2015, January 13\u201317). Perfaugur: Robust diagnostics for performance anomalies in cloud services. Proceedings of the 2015 IEEE 31st International Conference on Data Engineering, Seoul, Republic of Korea.","DOI":"10.1109\/ICDE.2015.7113365"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Nguyen, H., Tan, Y., and Gu, X. (2011, January 23\u201326). Pal: Propagation-aware anomaly localization for cloud hosted distributed applications. Proceedings of the Managing Large-Scale Systems via the Analysis of System Logs and the Application of Machine Learning Techniques, Cascais, Portugal.","DOI":"10.1145\/2038633.2038634"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Jayathilaka, H., Krintz, C., and Wolski, R. (2017, January 3\u20137). Performance monitoring and root cause analysis for cloud-hosted web applications. Proceedings of the 26th International Conference on World Wide Web, Perth, WA, Australia.","DOI":"10.1145\/3038912.3052649"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/4\/1919\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:28:24Z","timestamp":1760120904000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/4\/1919"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,8]]},"references-count":49,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["s23041919"],"URL":"https:\/\/doi.org\/10.3390\/s23041919","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,2,8]]}}}