{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T13:54:08Z","timestamp":1782395648790,"version":"3.54.5"},"reference-count":36,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2022,10,20]],"date-time":"2022-10-20T00:00:00Z","timestamp":1666224000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Centers of Academic Excellence in Cybersecurity","award":["2021 NCAE-C-002"],"award-info":[{"award-number":["2021 NCAE-C-002"]}]},{"name":"National Centers of Academic Excellence in Cybersecurity","award":["H98230-21-1-0170"],"award-info":[{"award-number":["H98230-21-1-0170"]}]},{"DOI":"10.13039\/100009226","name":"Cyber Research Innovation Grant Program","doi-asserted-by":"publisher","award":["2021 NCAE-C-002"],"award-info":[{"award-number":["2021 NCAE-C-002"]}],"id":[{"id":"10.13039\/100009226","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100009226","name":"Cyber Research Innovation Grant Program","doi-asserted-by":"publisher","award":["H98230-21-1-0170"],"award-info":[{"award-number":["H98230-21-1-0170"]}],"id":[{"id":"10.13039\/100009226","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>While computer networks and the massive amount of communication taking place on these networks grow, the amount of damage that can be done by network intrusions grows in tandem. The need is for an effective and scalable intrusion detection system (IDS) to address these potential damages that come with the growth of these networks. A great deal of contemporary research on near real-time IDS focuses on applying machine learning classifiers to labeled network intrusion datasets, but these datasets need be relevant pertaining to the currency of the network intrusions. This paper focuses on a newly created dataset, UWF-ZeekData22, that analyzes data from Zeek\u2019s Connection Logs collected using Security Onion 2 network security monitor and labelled using the MITRE ATT&amp;CK framework TTPs. Due to the volume of data, Spark, in the big data framework, was used to run many of the well-known classifiers (na\u00efve Bayes, random forest, decision tree, support vector classifier, gradient boosted trees, and logistic regression) to classify the reconnaissance and discovery tactics from this dataset. In addition to looking at the performance of these classifiers using Spark, scalability and response time were also analyzed.<\/jats:p>","DOI":"10.3390\/s22207999","type":"journal-article","created":{"date-parts":[[2022,10,21]],"date-time":"2022-10-21T00:34:30Z","timestamp":1666312470000},"page":"7999","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Detecting Reconnaissance and Discovery Tactics from the MITRE ATT&amp;CK Framework in Zeek Conn Logs Using Spark\u2019s Machine Learning in the Big Data Framework"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1886-4582","authenticated-orcid":false,"given":"Sikha","family":"Bagui","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of West Florida, Pensacola, FL 32514, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0106-3890","authenticated-orcid":false,"given":"Dustin","family":"Mink","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of West Florida, Pensacola, FL 32514, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Subhash","family":"Bagui","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Statistics, University of West Florida, Pensacola, FL 32514, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1914-711X","authenticated-orcid":false,"given":"Tirthankar","family":"Ghosh","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of West Florida, Pensacola, FL 32514, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tom","family":"McElroy","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of West Florida, Pensacola, FL 32514, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Esteban","family":"Paredes","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of West Florida, Pensacola, FL 32514, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nithisha","family":"Khasnavis","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of West Florida, Pensacola, FL 32514, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Russell","family":"Plenkers","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of West Florida, Pensacola, FL 32514, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"29696","DOI":"10.1109\/ACCESS.2021.3058528","article-title":"LocKedge: Low-Complexity Cyberattack Detection in IoT Edge Computing","volume":"9","author":"Huong","year":"2021","journal-title":"IEEE Access"},{"key":"ref_2","unstructured":"(2022, February 15). Zeek: About. The Zeek Project. Available online: https:\/\/zeek.org\/about\/."},{"key":"ref_3","unstructured":"(2020, August 20). University of West Florida. Available online: https:\/\/datasets.uwf.edu\/."},{"key":"ref_4","unstructured":"Trellix (2022, February 15). What Is the MITRE ATT&CK Framework?|Get the 101 Guide. Available online: https:\/\/www.trellix.com\/en-us\/security-awareness\/cybersecurity\/what-is-mitre-attack-framework.html."},{"key":"ref_5","unstructured":"MITRE ATT&CK (2022, February 16). Reconnaissance, Tactic TA0043\u2014Enterprise| MITRE ATT&CK\u00ae. Available online: https:\/\/attack.mitre.org\/tactics\/TA0043\/."},{"key":"ref_6","unstructured":"MITRE ATT&CK (2022, February 16). Discovery, Tactic TA0007\u2014Enterprise| MITRE ATT&CK\u00ae. Available online: https:\/\/attack.mitre.org\/tactics\/TA0007\/."},{"key":"ref_7","unstructured":"(2022, February 16). The Zeek Project. Base\/Protocols\/Conn\/Main.Zeek\u2014Book of Zeek (v5.0.0). Zeek. Available online: https:\/\/docs.zeek.org\/en\/v5.0.0\/scripts\/base\/protocols\/conn\/main.zeek.html."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Kala Karun, A., and Chitharanjan, K. (2013, January 11\u201312). A review on hadoop; HDFS infrastructure extensions. Proceedings of the 2013 IEEE Conference on Information and Communication Technologies, Thuckalay, India.","DOI":"10.1109\/CICT.2013.6558077"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.procs.2018.01.091","article-title":"Performance evaluation of intrusion detection based on machine learning using Apache Spark","volume":"127","author":"Belouch","year":"2018","journal-title":"Procedia Comput. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"824","DOI":"10.1016\/j.procs.2016.07.238","article-title":"A Framework for Fast and Efficient Cyber Security Network Intrusion Detection Using Apache Spark","volume":"93","author":"Gupta","year":"2016","journal-title":"Procedia Comput. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Morfino, V., and Rampone, S. (2020). Towards Near-Real-Time Intrusion Detection for IoT Devices using Supervised Learning and Apache Spark. Electronics, 9.","DOI":"10.3390\/electronics9030444"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"667","DOI":"10.1007\/s10586-017-0971-8","article-title":"A hybrid technique using binary particle swarm optimization and decision tree pruning for network intrusion detection","volume":"21","author":"Malik","year":"2018","journal-title":"Clust. Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1051","DOI":"10.1007\/s00521-016-2418-1","article-title":"An effective combining classifier approach using tree algorithms for network intrusion detection","volume":"28","author":"Kevric","year":"2017","journal-title":"Neural Comput. Appl."},{"key":"ref_14","first-page":"1","article-title":"Clustering Detection Method of Network Intrusion Feature Based on Support Vector Machine and LCA Block Algorithm","volume":"134","author":"Zhang","year":"2021","journal-title":"Wirel. Pers. Commun."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1007\/s11036-021-01838-x","article-title":"Support Vector Machine Intrusion Detection Scheme Based on Cloud-Fog Collaboration","volume":"27","author":"Du","year":"2022","journal-title":"Mob. Netw. Appl."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1186\/s40537-021-00426-w","article-title":"Detecting cybersecurity attacks across different network features and learners","volume":"8","author":"Leevy","year":"2021","journal-title":"J. Big Data"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sharafaldin, I., Habibi Lashkari, A., and Ghorbani, A.A. (2018, January 22\u201324). Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization. Proceedings of the 4th International Conference on Information Systems Security and Privacy, Madeira, Portugal.","DOI":"10.5220\/0006639801080116"},{"key":"ref_18","unstructured":"Microsoft (2022, April 08). Address Classes. Microsoft Docs. Available online: https:\/\/docs.microsoft.com\/en-us\/previous-versions\/windows\/it-pro\/windows-2000-server\/cc940018(v=technet.10)?redirectedfrom=MSDN."},{"key":"ref_19","first-page":"39","article-title":"Classifying UNSW-NB15 network traffic in the Big Data Framework using random forest in Spark","volume":"2","author":"Bagui","year":"2021","journal-title":"Int. J. Big Data Intell. Appl."},{"key":"ref_20","first-page":"25","article-title":"A machine learning approach to dataset imputation for software vulnerabilities","volume":"1284","author":"Rostami","year":"2020","journal-title":"Commun. Comput. Inf. Sci."},{"key":"ref_21","unstructured":"European Union Agency for Cybersecurity (2022, July 09). State of Vulnerabilities 2018\/2019\u2014Analysis of Events in the Life of Vulnerabilities. Available online: https:\/\/www.enisa.europa.eu\/publications\/technical-reports-on-cybersecurity-situation-the-state-of-cyber-security-vulnerabilities."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Al-Shaer, R., Spring, J.M., and Christou, E. (July, January 29). Learning the associations of MITRE ATT & CK Adversarial Techniques. Proceedings of the 2020 IEEE Conference on Communications and Network Security (CNS), Avignon, France.","DOI":"10.1109\/CNS48642.2020.9162207"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","article-title":"An Introduction to ROC Analysis (PDF)","volume":"27","author":"Fawcett","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_24","unstructured":"Apache Spark (2022, March 09). StringIndexer\u2014PySpark 3.3.0 Documentation. Available online: https:\/\/spark.apache.org\/docs\/latest\/api\/python\/reference\/api\/pyspark.ml.feature.StringIndexer.html."},{"key":"ref_25","unstructured":"Microsoft (2022, May 12). TCP\/IP Addressing and Subnetting\u2014Windows Client|Microsoft Docs. Available online: https:\/\/docs.microsoft.com\/en-us\/troubleshoot\/windows-client\/networking\/tcpip-addressing-and-subnetting."},{"key":"ref_26","unstructured":"Internet Assigned Numbers Authority (2022, May 12). Service Name and Transport Protocol Port Number Registry. Available online: https:\/\/www.iana.org\/assignments\/service-names-port-numbers\/service-names-port-numbers.xhtml."},{"key":"ref_27","unstructured":"Apache Spark (2022, March 09). Configuration\u2014Spark 3.3.0 Documentation. Available online: https:\/\/spark.apache.org\/docs\/latest\/configuration.html."},{"key":"ref_28","unstructured":"Oracle (2022, May 09). Securing Ports. Available online: https:\/\/docs.oracle.com\/cd\/E89228_03\/otn\/pdf\/install\/html_edmsc\/output\/chapter_6.htm."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1007\/s41060-016-0027-9","article-title":"Big data analytics on Apache Spark","volume":"1","author":"Salloum","year":"2016","journal-title":"Int. J. Data Sci. Anal."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/S0167-9473(01)00065-2","article-title":"Stochastic gradient boosting","volume":"38","author":"Friedman","year":"2002","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_31","unstructured":"Han, J., Kamber, M., and Pei, J. (2011). Data Mining: Concepts and Techniques, Morgan Kaufmann."},{"key":"ref_32","unstructured":"Accuracy Score (2022, August 06). Scikit-Learn. Available online: https:\/\/scikit-learn.org\/stable\/."},{"key":"ref_33","unstructured":"(2022, July 15). Available online: https:\/\/scikit-learn.org\/stable\/modules\/model_evaluation.html#accuracy-score."},{"key":"ref_34","unstructured":"Sasaki, Y. (2022, August 16). The Truth of the F-Measure (PDF). Available online: https:\/\/www.toyota-ti.ac.jp\/Lab\/Denshi\/COIN\/people\/yutaka.sasaki\/F-measure-YS-26Oct07.pdf."},{"key":"ref_35","first-page":"37","article-title":"Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation (PDF)","volume":"2","author":"Powers","year":"2011","journal-title":"J. Mach. Learn. Technol."},{"key":"ref_36","unstructured":"(2022, August 16). Available online: https:\/\/web.archive.org\/web\/20191114213255\/https:\/\/www.flinders.edu.au\/science_engineering\/fms\/School-CSEM\/publications\/tech_reps-research_artfcts\/TRRA_2007.pdf."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/20\/7999\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:57:56Z","timestamp":1760144276000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/20\/7999"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,20]]},"references-count":36,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["s22207999"],"URL":"https:\/\/doi.org\/10.3390\/s22207999","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,20]]}}}