{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T10:59:51Z","timestamp":1778929191904,"version":"3.51.4"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,10,22]],"date-time":"2021-10-22T00:00:00Z","timestamp":1634860800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,10,22]],"date-time":"2021-10-22T00:00:00Z","timestamp":1634860800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/M019462\/1"],"award-info":[{"award-number":["EP\/M019462\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010661","name":"Horizon 2020 Framework Programme","doi-asserted-by":"publisher","award":["700692"],"award-info":[{"award-number":["700692"]}],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2022,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Diverse layers of defence play an important role in the design of defence-in-depth architectures. The use of Intrusion Detection Systems (IDSs) are ubiquitous in this design. But the selection of the \u201cright\u201d IDSs in various configurations is an important decision that the security architects need to make. Additionally, the ability of these IDSs to adapt to the evolving threat-landscape also needs to be investigated. To help with these decisions, we need rigorous quantitative analysis. In this paper, we present a diversity analysis of open-source IDSs, Snort and Suricata, to help security architects tune\/deploy these IDSs. We analyse two types of diversities in these IDSs; configurational diversity and functional diversity. In the configurational diversity analysis, we investigate the diversity in the sets of rules and the Blacklisted IP Addresses (BIPAs) these IDSs use in their configurations. The functional diversity analysis investigates the differences in alerting behaviours of these IDSs when they analyse real network traffic, and how these differences evolve. The configurational diversity experiment utilises snapshots of the rules and BIPAs collected over a period of 5 months, from May to October 2017. The snapshots have been collected for three different off-the-shelf default configurations of the Snort IDS and the Emerging Threats (ET) configuration of the Suricata IDS. The functional diversity investigates the alerting behaviour of these two IDSs for a sample of the real network traffic collected in the same time window. Analysing the differences in these systems allows us to get insights into where the diversity in the behaviour of these systems comes from, how does it evolve and whether this has any effect on the alerting behaviour of these IDSs. This analysis gives insight to security architects on how they can combine and layer these systems in a defence-in-depth deployment.<\/jats:p>","DOI":"10.1007\/s10664-021-10046-w","type":"journal-article","created":{"date-parts":[[2021,10,22]],"date-time":"2021-10-22T11:03:42Z","timestamp":1634900622000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":30,"title":["Dynamical analysis of diversity in rule-based open source network intrusion detection systems"],"prefix":"10.1007","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8506-5721","authenticated-orcid":false,"given":"Hafizul","family":"Asad","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ilir","family":"Gashi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,10,22]]},"reference":[{"issue":"1","key":"10046_CR1","doi-asserted-by":"crossref","first-page":"e4150","DOI":"10.1002\/ett.4150","volume":"32","author":"Z Ahmad","year":"2021","unstructured":"Ahmad Z (2021) Network intrusion detection system: A systematic study of machine learning and deep learning approaches. Trans Emerg Telecommun Technol 32(1):e4150","journal-title":"Trans Emerg Telecommun Technol"},{"key":"10046_CR2","doi-asserted-by":"publisher","first-page":"102479","DOI":"10.1016\/j.jnca.2019.102479","volume":"150","author":"Alauthman M.","year":"2020","unstructured":"Alauthman M. et al (2020) An effcient reinforcement learning-based Botnet detection approach. J Netw Comput Appl 150:102479","journal-title":"J Netw Comput Appl"},{"key":"10046_CR3","doi-asserted-by":"crossref","unstructured":"Albin E, Rowe NC (2012) A realistic experimental comparison of the Suricata and Snort intrusion-detection systems. In: 2012 26th International conference on advanced information networking and applications workshops. IEEE, pp 122\u2013127","DOI":"10.1109\/WAINA.2012.29"},{"key":"10046_CR4","doi-asserted-by":"crossref","unstructured":"Alqahtani SM, John R (2016) A comparative study of different fuzzy classifiers for cloud intrusion detection systems\u2019 alerts. In: 2016 IEEE Symposium series on computational intelligence (SSCI). IEEE, pp 1\u20139","DOI":"10.1109\/SSCI.2016.7849911"},{"key":"10046_CR5","doi-asserted-by":"crossref","unstructured":"Algaith A et al (2017) Diversity with intrusion detection systems: An em- pirical study. In: 2017 IEEE 16th international symposium on network computing and applications (NCA). IEEE, pp 1\u20135","DOI":"10.1109\/NCA.2017.8171327"},{"key":"10046_CR6","doi-asserted-by":"crossref","unstructured":"Asad H, Gashi I (2018) Diversity in open source intrusion detection systems. In: International conference on computer safety, reliability, and security. Springer, pp 267\u2013281","DOI":"10.1007\/978-3-319-99130-6_18"},{"key":"10046_CR7","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1109\/MC.1984.1659219","volume":"8","author":"A Avizienis","year":"1984","unstructured":"Avizienis A, Kelly JPJ (1984) Fault tolerance by design diversity: Concepts and experiments. Computer 8:67\u201380","journal-title":"Computer"},{"key":"10046_CR8","unstructured":"Cummings JJ, Shirk Michael (2021) Pulledpork. https:\/\/github.com\/shirkdog\/pulledpork"},{"key":"10046_CR9","unstructured":"Emerging T. R. (2021) https:\/\/rules.emergingthreats.net\/open\/suricata\/. (visited on 04\/18\/2021)"},{"issue":"6","key":"10046_CR10","doi-asserted-by":"publisher","first-page":"735","DOI":"10.1002\/spe.2180","volume":"44","author":"M Garcia","year":"2014","unstructured":"Garcia M (2014) Analysis of operating system diversity for intrusion tolerance. Softw Pract Exper 44(6):735\u2013770","journal-title":"Softw Pract Exper"},{"key":"10046_CR11","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1016\/j.cose.2015.09.007","volume":"55","author":"P Garcia-Teodoro","year":"2015","unstructured":"Garcia-Teodoro P et al (2015) Automatic generation of HTTP intrusion signatures by selective identification of anomalies. Comput Secur 55:159\u2013174","journal-title":"Comput Secur"},{"key":"10046_CR12","doi-asserted-by":"crossref","unstructured":"Gupta V et al (2003) Dependability and performance evaluation of intrusion- tolerant server architectures. In: Latin-American symposium on dependable computing. Springer, pp 81\u2013101","DOI":"10.1007\/978-3-540-45214-0_9"},{"key":"10046_CR13","unstructured":"Hiltunen MA et al (2000) Survivability through customization and adapt- ability: The cactus approach. In: Proceedings DARPA information survivability conference and exposition, DISCEX\u201900, vol 1. IEEE, pp 294\u2013307"},{"issue":"3","key":"10046_CR14","doi-asserted-by":"publisher","first-page":"146","DOI":"10.1109\/MCOM.2012.6163595","volume":"50","author":"C-Y Ho","year":"2012","unstructured":"Ho C-Y, et al (2012) Statistical analysis of false positives and false negatives from real traffc with intrusion detection\/prevention systems. IEEE Commun Mag 50(3):146\u2013154.","journal-title":"IEEE Commun Mag"},{"key":"10046_CR15","first-page":"102426","volume":"51","author":"Q Hu","year":"2020","unstructured":"Hu Q, Yu S-Y, Asghar MR (2020) Analysing performance issues of open-source intrusion detection systems in high- speed networks. J Inf Secur Appl 51:102426","journal-title":"J Inf Secur Appl"},{"issue":"6","key":"10046_CR16","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1109\/MSP.2013.51","volume":"11","author":"S Kaur","year":"2013","unstructured":"Kaur S, Singh M (2013) Automatic attack signature generation systems: A review. IEEE Secur Privacy 11(6):54\u201361.","journal-title":"IEEE Secur Privacy"},{"key":"10046_CR17","doi-asserted-by":"crossref","unstructured":"Littlewood B, Strigini L (2004) Redundancy and diversity in security. In: European symposium on research in computer security. pp 423\u2013438. Springer","DOI":"10.1007\/978-3-540-30108-0_26"},{"key":"10046_CR18","doi-asserted-by":"publisher","first-page":"112963","DOI":"10.1016\/j.eswa.2019.112963","volume":"141","author":"M Lopez-Martin","year":"2020","unstructured":"Lopez-Martin M, Carro B, Sanchez-Esguevillas A (2020) Application of deep reinforcement learning to intrusion detection for supervised problems. Expert Syste Appl 141:112963","journal-title":"Expert Syste Appl"},{"key":"10046_CR19","unstructured":"MAFTIA R. P. (2003) http:\/\/research.cs.ncl.ac.uk\/cabernet\/www.laas.research.ec.org\/maftia\/ (visited on 04\/20\/2021)"},{"key":"10046_CR20","unstructured":"Majorczyk F, Totel \u00c9, M\u00e9 L (2007) Experiments on cots diversity as an intrusion detection and tolerance mechanism. In: Proceedings of the Fems (WRAITS 2007)"},{"issue":"1","key":"10046_CR21","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1145\/2808691","volume":"48","author":"A Milenkoski","year":"2015","unstructured":"Milenkoski A et al (2015) Evaluating computer intrusion detection systems: A survey of common practices. ACM Comput Surv (CSUR) 48(1):12","journal-title":"ACM Comput Surv (CSUR)"},{"key":"10046_CR22","doi-asserted-by":"crossref","unstructured":"Pathan A-SK (2014) The state of the art in intrusion prevention and detection CRC press","DOI":"10.1201\/b16390"},{"key":"10046_CR23","unstructured":"Pihelgas M (2012) A comparative analysis of open-source intrusion detection systems. Tallinn: Tallinn University of Technology and University of Tartu"},{"key":"10046_CR24","doi-asserted-by":"crossref","unstructured":"Reynolds J (2002) The design and implementation of an intrusion tolerant system. In: Proceedings international conference on dependable systems and networks. IEEE, pp 285\u2013290","DOI":"10.1109\/DSN.2002.1028912"},{"issue":"1","key":"10046_CR25","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1016\/j.jnca.2009.07.005","volume":"33","author":"K Salah","year":"2010","unstructured":"Salah K, Kahtani A (2010) Performance evaluation comparison of Snort NIDS under Linux and Windows Server. J Netw Comput Appl 33(1):6\u201315.","journal-title":"J Netw Comput Appl"},{"key":"10046_CR26","unstructured":"Sanders WH et al (2002) Probabilistic validation of intrusion tolerance. In: Supplemental volume int\u2019l conf. dependable systems and networks (DSN-2002) pp 78\u201379"},{"key":"10046_CR27","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1016\/j.future.2017.10.016","volume":"80","author":"SAR Shah","year":"2018","unstructured":"Shah SAR, Issac B (2018) Performance comparison of intrusion detection systems and application of machine learning to Snort system. Future Gener Comput Syst 80:157\u2013170","journal-title":"Future Gener Comput Syst"},{"key":"10046_CR28","unstructured":"Snort (2021) https:\/\/www.snort.org (visited on 04\/18\/2021)"},{"key":"10046_CR29","unstructured":"Snort B. (2021) https:\/\/talosintelligence.com\/documents\/ip-blacklist. (visited on 04\/18\/2021)"},{"key":"10046_CR30","unstructured":"Snort logs (2021) http:\/\/manual-snort-org.s3-website-us-east-1.amazonaws.com\/node21.html. (visited on 04\/18\/2021)"},{"key":"10046_CR31","unstructured":"Snort R. (2021) https:\/\/snort.org\/documents\/registered-vs-subscriber. (visited on 04\/18\/2021)"},{"key":"10046_CR32","unstructured":"Suricata logs (2021) https:\/\/suricata.readthedocs.io\/en\/suricata-.6.0.2\/output\/eve\/eve-json-output.html (visited on 04\/18\/2021)"},{"key":"10046_CR33","unstructured":"Suricata (2021) https:\/\/suricata-ids.org (visited on 04\/18\/2021)"},{"key":"10046_CR34","unstructured":"Suricata UT (2021) https:\/\/suricata-update.readthedocs.io\/en\/latest\/. (visited on 04\/18\/2021)"},{"key":"10046_CR35","doi-asserted-by":"crossref","unstructured":"Thongkanchorn K, Ngamsuriyaroj S, Visoottiviseth V (2013) Evaluation studies of three intrusion detection systems under various attacks and rule sets. In: 2013 IEEE international conference of IEEE Region 10 (TENCON 2013). IEEE, pp 1\u20134","DOI":"10.1109\/TENCON.2013.6718975"},{"issue":"4","key":"10046_CR36","doi-asserted-by":"publisher","first-page":"3639","DOI":"10.1109\/COMST.2019.2922584","volume":"21","author":"LN Tidjon","year":"2019","unstructured":"Tidjon LN, Frappier M, Mammar A (2019) Intrusion detection systems: A cross-domain overview. IEEE Commun Surv Tutor 21(4):3639\u20133681","journal-title":"IEEE Commun Surv Tutor"},{"key":"10046_CR37","unstructured":"van Niekerk B, Jacobs P (2015) ISACA JOURNAL"},{"key":"10046_CR38","doi-asserted-by":"crossref","unstructured":"Wang X et al (2013) Administrative evaluation of intrusion detection system. In: Proceedings of the 2nd annual conference on research in information technology. pp 47\u201352","DOI":"10.1145\/2512209.2512216"},{"key":"10046_CR39","unstructured":"Wazuh (2021) https:\/\/wazuh.com\/ (visited on 04\/18\/2021)"},{"key":"10046_CR40","unstructured":"Zeek (2021) https:\/\/docs.zeek.org\/en\/lts\/abou.html (visited on 04\/18\/2021)"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-10046-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-021-10046-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-10046-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,11]],"date-time":"2023-11-11T15:14:40Z","timestamp":1699715680000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-021-10046-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,22]]},"references-count":40,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1]]}},"alternative-id":["10046"],"URL":"https:\/\/doi.org\/10.1007\/s10664-021-10046-w","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,22]]},"assertion":[{"value":"1 September 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 October 2021","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"4"}}