{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T01:17:14Z","timestamp":1782782234869,"version":"3.54.5"},"reference-count":42,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2024,2,11]],"date-time":"2024-02-11T00:00:00Z","timestamp":1707609600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Network"],"abstract":"<jats:p>An intrusion detection system (IDS) perform postcompromise detection of security breaches whenever preventive measures such as firewalls do not avert an attack. However, these systems raise a vast number of alerts that must be analyzed and triaged by security analysts. This process is largely manual, tedious, and time-consuming. Alert correlation is a technique that reduces the number of intrusion alerts by aggregating alerts that are similar in some way. However, the correlation is performed outside the IDS through third-party systems and tools, after the IDS has already generated a high volume of alerts. These third-party systems add to the complexity of security operations. In this paper, we build on the highly researched area of alert and event correlation by developing a novel hierarchical event correlation model that promises to reduce the number of alerts issued by an intrusion detection system. This is achieved by correlating the events before the IDS classifies them. The proposed model takes the best features from similarity and graph-based correlation techniques to deliver an ensemble capability not possible by either approach separately. Further, we propose a correlation process for events rather than alerts as is the case in the current art. We further develop our own correlation and clustering algorithm which is tailor-made to the correlation and clustering of network event data. The model is implemented as a proof of concept with experiments run on standard intrusion detection sets. The correlation achieves an 87% data reduction through aggregation, producing nearly 21,000 clusters in about 30 s.<\/jats:p>","DOI":"10.3390\/network4010004","type":"journal-article","created":{"date-parts":[[2024,2,12]],"date-time":"2024-02-12T06:19:59Z","timestamp":1707718799000},"page":"68-90","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["A Hierarchical Security Event Correlation Model for Real-Time Threat Detection and Response"],"prefix":"10.3390","volume":"4","author":[{"given":"Herbert","family":"Maosa","sequence":"first","affiliation":[{"name":"Cyber Security Research Centre, London Metropolitan University, London N7 8DB, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7129-5809","authenticated-orcid":false,"given":"Karim","family":"Ouazzane","sequence":"additional","affiliation":[{"name":"Cyber Security Research Centre, London Metropolitan University, London N7 8DB, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7067-7848","authenticated-orcid":false,"given":"Mohamed Chahine","family":"Ghanem","sequence":"additional","affiliation":[{"name":"Cyber Security Research Centre, London Metropolitan University, London N7 8DB, UK"},{"name":"Department of Computer Science, University of Liverpool, Liverpool L69 3BX, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,11]]},"reference":[{"key":"ref_1","unstructured":"Amoroso, E.G. (2009). Intrusion Detection: An Introduction to Internet Surveillance, Correlation, Trace Back, Taps and Response, Intrusion.Net Books."},{"key":"ref_2","unstructured":"Feinstein, B., Curry, D., and Debar, H. (2007). The Intrusion Detection Message Exchange Format (IDMEF), SecureWorks Inc.. Internet Engineering Task Force, Request for Comments RFC 4765."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1289","DOI":"10.1016\/j.comnet.2012.10.022","article-title":"A model-based survey of alert correlation techniques","volume":"57","author":"Salah","year":"2013","journal-title":"Comput. Netw."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Sethi, A.S., Raynaud, Y., and Faure-Vincent, F. (1995). Integrated Network Management IV: Proceedings of the Fourth International Symposium on Integrated Network Management, 1995, Springer.","DOI":"10.1007\/978-0-387-34890-2"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1109\/TDSC.2004.21","article-title":"Comprehensive approach to intrusion detection alert correlation","volume":"1","author":"Valeur","year":"2004","journal-title":"IEEE Trans. Dependable Secure Comput."},{"key":"ref_6","unstructured":"Cuppens, F., and Miege, A. (2002, January 12\u201315). Alert correlation in a cooperative intrusion detection framework. Proceedings of the 2002 IEEE Symposium on Security and Privacy, Berkeley, CA, USA."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Tian, D., Changzhen, H., Qi, Y., and Jianqiao, W. (2009, January 18\u201320). Hierarchical Distributed Alert Correlation Model. Proceedings of the 2009 Fifth International Conference on Information Assurance and Security, Xi\u2019an, China.","DOI":"10.1109\/IAS.2009.26"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Patton, R.M., Beaver, J.M., Steed, C.A., Potok, T.E., and Treadwell, J.N. (2011, January 4\u20138). Hierarchical clustering and visualization of aggregate cyber data. Proceedings of the 2011 7th International Wireless Communications and Mobile Computing Conference, Istanbul, Turkey.","DOI":"10.1109\/IWCMC.2011.5982725"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Diakhame, M.L., Diallo, C., and Mejri, M. (2023, January 16\u201318). MCM-CASR: Novel Alert Correlation Framework for Cyber Attack Scenario Reconstruction Based on NLP, NER, and Semantic Similarity. Proceedings of the 2023 7th Cyber Security in Networking Conference (CSNet), Montreal, QC, Canada.","DOI":"10.1109\/CSNet59123.2023.10339751"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1257\/jel.49.1.72","article-title":"Frontiers of Real-Time Data Analysis","volume":"49","author":"Croushore","year":"2011","journal-title":"J. Econ. Lit."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Lin, Z., Li, S., and Ma, Y. (2010, January 23\u201325). Real-Time Intrusion Alert Correlation System Based on Prerequisites and Consequence. Proceedings of the 2010 6th International Conference on Wireless Communications Networking and Mobile Computing (WiCOM), Chengdu, China.","DOI":"10.1109\/WICOM.2010.5601285"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Lopez, M.A., Lobato, A.G.P., Duarte, O.C.M.B., and Pujolle, G. (2018, January 24\u201325). An evaluation of a virtual network function for real-time threat detection using stream processing. Proceedings of the 2018 Fourth International Conference on Mobile and Secure Services (MobiSecServ), Miami Beach, FL, USA.","DOI":"10.1109\/MOBISECSERV.2018.8311440"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Dong, Y., Wang, R., and He, J. (2019, January 18\u201320). Real-Time Network Intrusion Detection System Based on Deep Learning. Proceedings of the 2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS), Beijing, China.","DOI":"10.1109\/ICSESS47205.2019.9040718"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4018\/IJISP.311465","article-title":"Real-time cyber analytics data collection framework","volume":"16","author":"Maosa","year":"2022","journal-title":"Int. J. Inf. Secur. Priv. IJISP"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Kumari, V.P. (2016, January 24\u201326). Real time streaming fastdata and proposed framework for disaster alerts. Proceedings of the 2016 International Conference on Emerging Trends in Engineering, Technology and Science (ICETETS), Pudukkottai, India.","DOI":"10.1109\/ICETETS.2016.7603009"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ma, J., Li, Z., and Li, W. (2008, January 18\u201320). Real-Time Alert Stream Clustering and Correlation for Discovering Attack Strategies. Proceedings of the 2008 Fifth International Conference on Fuzzy Systems and Knowledge Discovery, Jinan, China.","DOI":"10.1109\/FSKD.2008.522"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ramaki, A.A., Khosravi-Farmad, M., and Bafghi, A.G. (2015, January 8\u201310). Real time alert correlation and prediction using Bayesian networks. Proceedings of the 2015 12th International Iranian Society of Cryptology Conference on Information Security and Cryptology (ISCISC), Rasht, Iran.","DOI":"10.1109\/ISCISC.2015.7387905"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1109\/JAS.2020.1003099","article-title":"A real-time and ubiquitous network attack detection based on deep belief network and support vector machine","volume":"7","author":"Zhang","year":"2020","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Li, B., and Chan, K.C.C. (2014, January 27\u201329). A fast big data collection system using MapReduce framework. Proceedings of the 2014 IEEE 3rd International Conference on Cloud Computing and Intelligence Systems, Shenzhen, China.","DOI":"10.1109\/CCIS.2014.7175793"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1145\/3325061.3325062","article-title":"On the alert correlation process for the detection of multi-step attacks and a graph-based realization","volume":"19","author":"Haas","year":"2019","journal-title":"ACM SIGAPP Appl. Comput. Rev."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1659","DOI":"10.1016\/j.eswa.2007.01.040","article-title":"DDoS attack detection method using cluster analysis","volume":"34","author":"Lee","year":"2008","journal-title":"Expert Syst. Appl."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Qin, X., Xu, T., and Wang, C. (2015, January 19\u201320). DDoS Attack Detection Using Flow Entropy and Clustering Technique. Proceedings of the 2015 11th International Conference on Computational Intelligence and Security (CIS), Shenzhen, China.","DOI":"10.1109\/CIS.2015.105"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"814","DOI":"10.1038\/nature03607","article-title":"Uncovering the overlapping community structure of complex networks in nature and society","volume":"435","author":"Palla","year":"2005","journal-title":"Nature"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1109\/TNN.2005.845141","article-title":"Survey of clustering algorithms","volume":"16","author":"Xu","year":"2005","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1023\/B:DAMI.0000005258.31418.83","article-title":"Mining frequent patterns without candidate generation: A frequent-pattern tree approach","volume":"8","author":"Han","year":"2004","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_26","unstructured":"Vaarandi, R. (2003, January 3). A data clustering algorithm for mining patterns from event logs. Proceedings of the 3rd IEEE Workshop on IP Operations and Management (IPOM 2003) (IEEE Cat. No.03EX764), Kansas City, MO, USA."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1016\/j.cose.2008.05.005","article-title":"Building network attack graph for alert causal correlation","volume":"27","author":"Zhang","year":"2008","journal-title":"Comput. Secur."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2917","DOI":"10.1016\/j.comcom.2006.04.001","article-title":"Using attack graphs for correlating, hypothesizing, and predicting intrusion alerts","volume":"29","author":"Wang","year":"2006","journal-title":"Comput. Commun."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1288","DOI":"10.1109\/TIFS.2012.2191963","article-title":"Anomaly Detection Using Proximity Graph and PageRank Algorithm","volume":"7","author":"Yao","year":"2012","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"904","DOI":"10.1109\/TITS.2022.3140229","article-title":"A Novel Spatial-Temporal Multi-Scale Alignment Graph Neural Network Security Model for Vehicles Prediction","volume":"24","author":"Diao","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_31","unstructured":"Lundin, E., and Jonsson, E. (2002). Survey of Intrusion Detection Research, Chalmers University of Technology."},{"key":"ref_32","unstructured":"Roesch, M. (1999, January 7\u201312). Snort: Lightweight intrusion detection for networks. Proceedings of the LISA \u201999: Proceedings of the 13th USENIX Conference on System Administration, Seattle, WA, USA."},{"key":"ref_33","unstructured":"Ning, P., Xu, D., Healey, C.G., and Amant, R.S. Building Attack Scenarios through Integration of Complementary Alert Correlation Method. Proceedings of the 10th Annual Network and Distributed System Security Symposium (NDSS \u201904), 2004, Available online: https:\/\/healey.csc.ncsu.edu\/publications\/15812-building-attack-scenarios-through-integration-of-complementary-alert-correlation-method."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Schueller, Q., Basu, K., Younas, M., Patel, M., and Ball, F. (2018, January 21\u201323). A Hierarchical Intrusion Detection System using Support Vector Machine for SDN Network in Cloud Data Center. Proceedings of the 2018 28th International Telecommunication Networks and Applications Conference (ITNAC), Sydney, NSW, Australia.","DOI":"10.1109\/ATNAC.2018.8615255"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1186\/s42400-019-0038-7","article-title":"Survey of intrusion detection systems: Techniques, datasets and challenges","volume":"2","author":"Khraisat","year":"2019","journal-title":"Cybersecurity"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1564","DOI":"10.1109\/LCOMM.2020.3048995","article-title":"Discovering Attack Scenarios via Intrusion Alert Correlation Using Graph Convolutional Networks","volume":"25","author":"Cheng","year":"2021","journal-title":"IEEE Commun. Lett."},{"key":"ref_37","unstructured":"Ghanem, M.C. (2022). Towards an Efficient Automation of Network Penetration Testing Using Model-Based Reinforcement Learning. [Doctoral Dissertation, University of London]."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1016\/j.cose.2014.10.006","article-title":"RTECA: Real time episode correlation algorithm for multi-step attack scenarios detection","volume":"49","author":"Ramaki","year":"2015","journal-title":"Comput. Secur."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Pakhira, M.K. (2014, January 14\u201316). A Linear Time-Complexity k-Means Algorithm Using Cluster Shifting. Proceedings of the 2014 International Conference on Computational Intelligence and Communication Networks, Bhopal, India.","DOI":"10.1109\/CICN.2014.220"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.tcs.2006.06.015","article-title":"The worst-case time complexity for generating all maximal cliques and computational experiments","volume":"363","author":"Tomita","year":"2006","journal-title":"Theor. Comput. Sci."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Wang, X., Gong, X., Yu, L., and Liu, J. (2021, January 20\u201322). MAAC: Novel alert correlation method to detect multi-step attack. Proceedings of the 2021 IEEE 20th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Shenyang, China.","DOI":"10.1109\/TrustCom53373.2021.00106"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Anbarestani, R., Akbari, B., and Fathi, F. (2012, January 15\u201317). An iterative alert correlation method for extracting network intrusion scenarios. Proceedings of the 20th Iranian Conference on Electrical Engineering (ICEE2012), Tehran, Iran.","DOI":"10.1109\/IranianCEE.2012.6292441"}],"container-title":["Network"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2673-8732\/4\/1\/4\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:58:33Z","timestamp":1760104713000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2673-8732\/4\/1\/4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,11]]},"references-count":42,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["network4010004"],"URL":"https:\/\/doi.org\/10.3390\/network4010004","relation":{},"ISSN":["2673-8732"],"issn-type":[{"value":"2673-8732","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,11]]}}}