{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,13]],"date-time":"2026-07-13T18:20:43Z","timestamp":1783966843492,"version":"3.55.0"},"reference-count":46,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,1,26]],"date-time":"2025-01-26T00:00:00Z","timestamp":1737849600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"U.S.-Israel Energy Center managed by the Israel-U.S. Binational Industrial Research and Development (BIRD) Foundation"},{"name":"Fujitsu"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>Analysts in Security Operations Centers (SOCs) are often occupied with time-consuming investigations of alerts from Network Intrusion Detection Systems (NIDSs). Many NIDS rules lack clear explanations and associations with attack techniques, complicating the alert triage and the generation of attack hypotheses. Large Language Models (LLMs) may be a promising technology to reduce the alert explainability gap by associating rules with attack techniques. In this paper, we investigate the ability of three prominent LLMs (ChatGPT, Claude, and Gemini) to reason about NIDS rules while labeling them with MITRE ATT&amp;CK tactics and techniques. We discuss prompt design and present experiments performed with 973 Snort rules. Our results indicate that while LLMs provide explainable, scalable, and efficient initial mappings, traditional machine learning (ML) models consistently outperform them in accuracy, achieving higher precision, recall, and F1-scores. These results highlight the potential for hybrid LLM-ML approaches to enhance SOC operations and better address the evolving threat landscape. By utilizing automation, the presented methods will enhance the analysis efficiency of SOC alerts, and decrease workloads for analysts.<\/jats:p>","DOI":"10.3390\/bdcc9020023","type":"journal-article","created":{"date-parts":[[2025,1,27]],"date-time":"2025-01-27T06:39:51Z","timestamp":1737959991000},"page":"23","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Labeling Network Intrusion Detection System (NIDS) Rules with MITRE ATT&amp;CK Techniques: Machine Learning vs. Large Language Models"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5310-2324","authenticated-orcid":false,"given":"Nir","family":"Daniel","sequence":"first","affiliation":[{"name":"Department of Software and Information Systems Engineering, Ben-Gurion University, Beer Sheva 8410501, Israel"},{"name":"Cyber@BGU, Cyber Labs at Ben-Gurion University, Beer Sheva 8410501, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Florian Klaus","family":"Kaiser","sequence":"additional","affiliation":[{"name":"Agnostic Intelligence AG, 6300 Zug, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shay","family":"Giladi","sequence":"additional","affiliation":[{"name":"Department of Software and Information Systems Engineering, Ben-Gurion University, Beer Sheva 8410501, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sapir","family":"Sharabi","sequence":"additional","affiliation":[{"name":"Department of Software and Information Systems Engineering, Ben-Gurion University, Beer Sheva 8410501, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Raz","family":"Moyal","sequence":"additional","affiliation":[{"name":"Department of Software and Information Systems Engineering, Ben-Gurion University, Beer Sheva 8410501, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shalev","family":"Shpolyansky","sequence":"additional","affiliation":[{"name":"Department of Software and Information Systems Engineering, Ben-Gurion University, Beer Sheva 8410501, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andres","family":"Murillo","sequence":"additional","affiliation":[{"name":"Fujitsu Ltd., Kawasaki 212-0014, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0918-0146","authenticated-orcid":false,"given":"Aviad","family":"Elyashar","sequence":"additional","affiliation":[{"name":"Cyber@BGU, Cyber Labs at Ben-Gurion University, Beer Sheva 8410501, Israel"},{"name":"Department of Computer Science, Shamoon College of Engineering, Ashdod 77245, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7229-3899","authenticated-orcid":false,"given":"Rami","family":"Puzis","sequence":"additional","affiliation":[{"name":"Department of Software and Information Systems Engineering, Ben-Gurion University, Beer Sheva 8410501, Israel"},{"name":"Cyber@BGU, Cyber Labs at Ben-Gurion University, Beer Sheva 8410501, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,1,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chakrabarti, S., Chakraborty, M., and Mukhopadhyay, I. (2010, January 23). Study of snort-based IDS. Proceedings of the International Conference and Workshop on Emerging Trends in Technology, Almaty, Kazakhstan.","DOI":"10.1145\/1741906.1741914"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Elitzur, A., Puzis, R., and Zilberman, P. (2019, January 15). Attack hypothesis generation. Proceedings of the 2019 European Intelligence and Security Informatics Conference (EISIC), Warsaw, Poland.","DOI":"10.1109\/EISIC49498.2019.9108886"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Lin, S.X., Li, Z.J., Chen, T.Y., and Wu, D.J. (2022, January 20). Attack tactic labeling for cyber threat hunting. Proceedings of the 2022 24th International Conference on Advanced Communication Technology (ICACT), Seoul, South Korea.","DOI":"10.23919\/ICACT53585.2022.9728949"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Bagui, S.S., Mink, D., Bagui, S.C., Ghosh, T., Plenkers, R., McElroy, T., Dulaney, S., and Shabanali, S. (2023). Introducing UWF-ZeekData22: A Comprehensive Network Traffic Dataset Based on the MITRE ATT&CK Framework. Data, 8.","DOI":"10.3390\/data8010018"},{"key":"ref_5","unstructured":"Sentonas, M. (2024, August 10). Crowdstrike Introduces Charlotte AI, Generative AI Security Analyst\u2014Crowdstrike, 2023. Available online: https:\/\/www.crowdstrike.com\/en-us\/blog\/crowdstrike-introduces-charlotte-ai-to-deliver-generative-ai-powered-cybersecurity\/."},{"key":"ref_6","unstructured":"T\u00f6rnberg, P. (2023). Chatgpt-4 outperforms experts and crowd workers in annotating political twitter messages with zero-shot learning. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Long, C., Lowe, K., dos Santos, A., Zhang, J., Alanazi, A., O\u2019Brien, D., Wright, E., and Cote, D. (2023). Evaluating ChatGPT4 in Canadian Otolaryngology-Head and Neck Surgery Board Examination using the CVSA Model. medRxiv, 2023\u201305.","DOI":"10.1101\/2023.05.30.23290758"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"102810","DOI":"10.1016\/j.cose.2022.102810","article-title":"Datasets are not enough: Challenges in labeling network traffic","volume":"120","author":"Guerra","year":"2022","journal-title":"Comput. Secur."},{"key":"ref_9","unstructured":"Gjerstad, J.L. (2022). Generating Labelled Network Datasets of APT with the MITRE CALDERA Framework. [Master\u2019s Thesis, University of Oslo]."},{"key":"ref_10","unstructured":"Palacin, V. (2021). Practical Threat Intelligence and Data-Driven Threat Hunting, Packt Publishing."},{"key":"ref_11","unstructured":"Chismon, D., and Ruks, M. (2015). Threat Intelligence: Collecting, Analysing, Evaluating, MWR Infosecurity Ltd."},{"key":"ref_12","unstructured":"Haddad, A., Aaraj, N., Nakov, P., and Mare, S.F. (2023). Automated Mapping of CVE Vulnerability Records to MITRE CWE Weaknesses. arXiv."},{"key":"ref_13","unstructured":"Daszczyszak, R., Ellis, D., Luke, S., and Whitley, S. (2019). Ttp-Based Hunting, Mitre Corp Mclean. Technical Report."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4793","DOI":"10.1109\/TDSC.2022.3233703","article-title":"Attack Hypotheses Generation Based on Threat Intelligence Knowledge Graph","volume":"20","author":"Kaiser","year":"2023","journal-title":"IEEE Trans. Dependable Secur. Comput."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Liao, X., Yuan, K., Wang, X., Li, Z., Xing, L., and Beyah, R. (2016, January 24\u201328). Acing the IOC game: Toward automatic discovery and analysis of open-source cyber threat intelligence. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria.","DOI":"10.1145\/2976749.2978315"},{"key":"ref_16","unstructured":"Strom, B.E., Applebaum, A., Miller, D.P., Nickels, K.C., Pennington, A.G., and Thomas, C.B. (2020). MITRE ATT&CK\u00ae: Design and Philosophy, The MITRE Corporation."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Landauer, M., Frank, M., Skopik, F., Hotwagner, W., Wurzenberger, M., and Rauber, A. (2022, January 27). A framework for automatic labeling of log datasets from model-driven testbeds for HIDS evaluation. Proceedings of the 2022 ACM Workshop on Secure and Trustworthy Cyber-Physical Systems, Baltimore, MD, USA.","DOI":"10.1145\/3510547.3517924"},{"key":"ref_18","first-page":"444","article-title":"Survey on intrusion detection system types","volume":"7","author":"Othman","year":"2018","journal-title":"Int. J. Cyber-Secur. Digit. Forensics"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Peng, Y., and Wang, H. (2012, January 17\u201319). Design and implementation of network instruction detection system based on snort and NTOP. Proceedings of the 2012 International Conference on Systems and Informatics (ICSAI2012), Beijing, China.","DOI":"10.1109\/ICSAI.2012.6223247"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Khamphakdee, N., Benjamas, N., and Saiyod, S. (2014, January 14\u201317). Improving intrusion detection system based on Snort rules for network probe attack detection. Proceedings of the 2014 2nd International Conference on Information and Communication Technology (ICoICT), Berlin, Germany.","DOI":"10.1109\/ICoICT.2014.6914042"},{"key":"ref_21","unstructured":"Motlagh, F.N., Hajizadeh, M., Majd, M., Najafi, P., Cheng, F., and Meinel, C. (2024). Large language models in cybersecurity: State-of-the-art. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Tod-R\u0103ileanu, G., and Axinte, S.D. (2023, January 25\u201329). ChatGPT-Information Security Overview. Proceedings of the International Conference on Cybersecurity and Cybercrime, Porto, Portugal.","DOI":"10.19107\/CYBERCON.2023.10"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liu, Y., Tao, S., Meng, W., Yao, F., Zhao, X., and Yang, H. (2024, January 13\u201316). Logprompt: Prompt engineering towards zero-shot and interpretable log analysis. Proceedings of the 2024 IEEE\/ACM 46th International Conference on Software Engineering: Companion Proceedings, Hong Kong, China.","DOI":"10.1145\/3639478.3643108"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"53972","DOI":"10.1109\/ACCESS.2022.3176098","article-title":"Generating labeled training datasets towards unified network intrusion detection systems","volume":"10","author":"Ishibashi","year":"2022","journal-title":"IEEE Access"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Husari, G., Al-Shaer, E., Ahmed, M., Chu, B., and Niu, X. (2017, January 4\u20138). Ttpdrill: Automatic and accurate extraction of threat actions from unstructured text of CTI sources. Proceedings of the 33rd Annual Computer Security Applications, Orlando, FL, USA.","DOI":"10.1145\/3134600.3134646"},{"key":"ref_26","unstructured":"Legoy, V., Caselli, M., Seifert, C., and Peter, A. (2020). Automated retrieval of ATT&CK tactics and techniques for cyber threat reports. arXiv."},{"key":"ref_27","unstructured":"Mendsaikhan, O., Hasegawa, H., Yamaguchi, Y., and Shimada, H. (2020, January 21\u201325). Automatic mapping of vulnerability information to adversary techniques. Proceedings of the The Fourteenth International Conference on Emerging Security Information, Systems and Technologies SECUREWARE2020, Valencia, Spain."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Satvat, K., Gjomemo, R., and Venkatakrishnan, V. (July, January 30). Extractor: Extracting attack behavior from threat reports. Proceedings of the 2021 IEEE European Symposium on Security and Privacy (EuroS&P), Venice, Italy.","DOI":"10.1109\/EuroSP51992.2021.00046"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1186\/s42400-021-00106-5","article-title":"TIM: Threat context-enhanced TTP intelligence mining on unstructured threat data","volume":"5","author":"You","year":"2022","journal-title":"Cybersecurity"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Li, Z., Zeng, J., Chen, Y., and Liang, Z. (2022, January 26\u201330). AttacKG: Constructing technique knowledge graph from cyber threat intelligence reports. Proceedings of the Computer Security\u2013ESORICS 2022: 27th European Symposium on Research in Computer Security, Copenhagen, Denmark. Proceedings, Part I.","DOI":"10.1007\/978-3-031-17140-6_29"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Rani, N., Saha, B., Maurya, V., and Shukla, S.K. (February, January 30). TTPHunter: Automated Extraction of Actionable Intelligence as TTPs from Narrative Threat Reports. Proceedings of the 2023 Australasian Computer Science Week, Melbourne, VIC, Australia.","DOI":"10.1145\/3579375.3579391"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Aghaei, E., Niu, X., Shadid, W., and Al-Shaer, E. (2022, January 17\u201319). Securebert: A domain-specific language model for cybersecurity. Proceedings of the International Conference on Security and Privacy in Communication Systems, Virtual Event.","DOI":"10.1007\/978-3-031-25538-0_3"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Gabrys, R., Bilinski, M., Fugate, S., and Silva, D. (2024, January 8\u201310). Using Natural Language Processing Tools to Infer Adversary Techniques and Tactics Under the Mitre ATT&CK Framework. Proceedings of the 2024 IEEE 14th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA.","DOI":"10.1109\/CCWC60891.2024.10427746"},{"key":"ref_34","unstructured":"McPhee, M. (2020). Methods to Employ Zeek in Detecting MITRE ATT&CK Techniques, The MITRE Corporation."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Arafune, M., Rajalakshmi, S., Jaldon, L., Jadidi, Z., Pal, S., Foo, E., and Venkatachalam, N. (2022, January 26\u201329). Design and development of automated threat hunting in industrial control systems. Proceedings of the 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), Las Vegas, NV, USA.","DOI":"10.1109\/PerComWorkshops53856.2022.9767375"},{"key":"ref_36","unstructured":"Garcia, S., and Valeros, V. (2023). Towards a better labeling process for network security datasets. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Masumi, K., Han, C., Ban, T., and Takahashi, T. (2021, January 26\u201328). Towards efficient labeling of network incident datasets using tcpreplay and snort. Proceedings of the Eleventh ACM Conference on Data and Application Security and Privacy, Virtual.","DOI":"10.1145\/3422337.3450326"},{"key":"ref_38","unstructured":"Sharma, Y., Birnbach, S., and Martinovic, I. (2022). RADAR: Effective Network-based Malware Detection based on the MITRE ATT&CK Framework. arXiv."},{"key":"ref_39","first-page":"2","article-title":"ChatIDS: Advancing Explainable Cybersecurity Using Generative AI","volume":"17","author":"Grimmer","year":"2024","journal-title":"Int. J. Adv. Secur."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Daniel, N., Kaiser, F.K., Dzega, A., Elyashar, A., and Puzis, R. (2023, January 25\u201329). Labeling NIDS Rules with MITRE ATT &CK Techniques Using ChatGPT. Proceedings of the European Symposium on Research in Computer Security, The Hague, The Netherlands.","DOI":"10.1007\/978-3-031-54129-2_5"},{"key":"ref_41","unstructured":"Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., and Yang, Y. (2024). Self-refine: Iterative refinement with self-feedback. Adv. Neural Inf. Process. Syst., 36."},{"key":"ref_42","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"Openai Blog"},{"key":"ref_43","unstructured":"Brown, T.B. (2020). Language models are few-shot learners. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., and Chen, W. (2021). What Makes Good In-Context Examples for GPT-3?. arXiv.","DOI":"10.18653\/v1\/2022.deelio-1.10"},{"key":"ref_45","unstructured":"Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H.P.D.O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., and Brockman, G. (2021). Evaluating large language models trained on code. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Wong, M.F., Guo, S., Hang, C.N., Ho, S.W., and Tan, C.W. (2023). Natural language generation and understanding of big code for AI-assisted programming: A review. Entropy, 25.","DOI":"10.3390\/e25060888"}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/2\/23\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T10:36:29Z","timestamp":1759919789000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/2\/23"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,26]]},"references-count":46,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,2]]}},"alternative-id":["bdcc9020023"],"URL":"https:\/\/doi.org\/10.3390\/bdcc9020023","relation":{},"ISSN":["2504-2289"],"issn-type":[{"value":"2504-2289","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,26]]}}}