{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T10:35:28Z","timestamp":1781606128849,"version":"3.54.5"},"reference-count":37,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,3,30]],"date-time":"2025-03-30T00:00:00Z","timestamp":1743292800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>This paper explores the potential use of Large Language Models (LLMs), such as ChatGPT, Google Gemini, and Microsoft Copilot, in threat hunting, specifically focusing on Living off the Land (LotL) techniques. LotL methods allow threat actors to blend into regular network activity, which makes detection by automated security systems challenging. The study seeks to determine whether LLMs can reliably generate effective queries for security tools, enabling organisations with limited budgets and expertise to conduct threat hunting. A testing environment was created to simulate LotL techniques, and LLM-generated queries were used to identify malicious activity. The results demonstrate that LLMs do not consistently produce accurate or reliable queries for detecting these techniques, particularly for users with varying skill levels. However, while LLMs may not be suitable as standalone tools for threat hunting, they can still serve as supportive resources within a broader security strategy. These findings suggest that, although LLMs offer potential, they should not be relied upon for accurate results in threat detection and require further refinement to be effectively integrated into cybersecurity workflows.<\/jats:p>","DOI":"10.3390\/make7020031","type":"journal-article","created":{"date-parts":[[2025,4,1]],"date-time":"2025-04-01T10:59:59Z","timestamp":1743505199000},"page":"31","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Leveraging LLMs for Non-Security Experts in Threat Hunting: Detecting Living off the Land Techniques"],"prefix":"10.3390","volume":"7","author":[{"given":"Antreas","family":"Konstantinou","sequence":"first","affiliation":[{"name":"Blockpass ID Lab, Edinburgh Napier University, Edinburgh EH10 5DT, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-2036-426X","authenticated-orcid":false,"given":"Dimitrios","family":"Kasimatis","sequence":"additional","affiliation":[{"name":"Blockpass ID Lab, Edinburgh Napier University, Edinburgh EH10 5DT, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0809-3523","authenticated-orcid":false,"given":"William J.","family":"Buchanan","sequence":"additional","affiliation":[{"name":"Blockpass ID Lab, Edinburgh Napier University, Edinburgh EH10 5DT, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3950-4719","authenticated-orcid":false,"given":"Sana Ullah","family":"Jan","sequence":"additional","affiliation":[{"name":"Blockpass ID Lab, Edinburgh Napier University, Edinburgh EH10 5DT, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6289-8248","authenticated-orcid":false,"given":"Jawad","family":"Ahmad","sequence":"additional","affiliation":[{"name":"Cybersecurity Center, Prince Mohammad Bin Fahd University, Al-Khobar 34754, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-2882-6452","authenticated-orcid":false,"given":"Ilias","family":"Politis","sequence":"additional","affiliation":[{"name":"Industrial Systems Institute, Research Center \u201cATHENA\u201d, Patras Science Park Building, Platani, 265 04 Patras, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3392-9970","authenticated-orcid":false,"given":"Nikolaos","family":"Pitropakis","sequence":"additional","affiliation":[{"name":"Blockpass ID Lab, Edinburgh Napier University, Edinburgh EH10 5DT, UK"},{"name":"Department of Information Technology, The American College of Greece, 153 42 Athens, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"100211","DOI":"10.1016\/j.hcc.2024.100211","article-title":"A survey on large language model (llm) security and privacy: The good, the bad, and the ugly","volume":"4","author":"Yao","year":"2024","journal-title":"High-Confid. Comput."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Rawindaran, N., Jayal, A., and Prakash, E. (2021). Machine learning cybersecurity adoption in small and medium enterprises in developed countries. Computers, 10.","DOI":"10.3390\/computers10110150"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Barr-Smith, F., Ugarte-Pedrero, X., Graziano, M., Spolaor, R., and Martinovic, I. (2021, January 24\u201327). Survivalism: Systematic analysis of windows malware living-off-the-land. Proceedings of the 2021 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA.","DOI":"10.1109\/SP40001.2021.00047"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1007\/s10207-014-0255-8","article-title":"Behaviour reflects personality: Detecting co-residence attacks on Xen-based cloud environments","volume":"14","author":"Pitropakis","year":"2015","journal-title":"Int. J. Inf. Secur."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ussath, M., Jaeger, D., Cheng, F., and Meinel, C. (2016, January 16\u201318). Advanced persistent threats: Behind the scenes. Proceedings of the 2016 Annual Conference on Information Science and Systems (CISS), Princeton, NJ, USA.","DOI":"10.1109\/CISS.2016.7460498"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"61695","DOI":"10.1109\/ACCESS.2022.3181278","article-title":"Utilizing cyber threat hunting techniques to find ransomware attacks: A survey of the state of the art","volume":"10","author":"Aldauiji","year":"2022","journal-title":"IEEE Access"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"85701","DOI":"10.1109\/ACCESS.2022.3197899","article-title":"A survey on the cyber security of small-to-medium businesses: Challenges, research focus and recommendations","volume":"10","author":"Chidukwani","year":"2022","journal-title":"IEEE Access"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"100162","DOI":"10.1016\/j.iot.2020.100162","article-title":"Identifying the attack surface for IoT network","volume":"9","author":"Rizvi","year":"2020","journal-title":"Internet Things"},{"key":"ref_9","unstructured":"CrowdStrike (2024, December 22). WIZARD SPIDER Update: Resilient, Reactive and Resolute. Available online: https:\/\/www.crowdstrike.com\/en-us\/blog\/wizard-spider-adversary-update\/."},{"key":"ref_10","unstructured":"IBM (2024, January 08). What Is Threat Hunting?. Available online: https:\/\/www.ibm.com\/think\/topics\/threat-hunting."},{"key":"ref_11","unstructured":"Gunter, D., and Seitz, M. (2024, January 08). A Practical Model for Conducting Cyber Threat Hunting. SANS, Available online: https:\/\/www.sans.org\/white-papers\/38710\/."},{"key":"ref_12","unstructured":"Fuchs, M., and Lemon, J. (2019). Sans 2019 Threat Hunting Survey: The Differing Needs of New and Experienced Hunters, SANS Institute Information Reading Room."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"101769","DOI":"10.1016\/j.techsoc.2021.101769","article-title":"The cybersecurity labour shortage in Europe: Moving to a new concept for education and training","volume":"67","year":"2021","journal-title":"Technol. Soc."},{"key":"ref_14","unstructured":"Central Digital and Data Office (2024, December 19). Detecting the Unknown: A Guide to Threat Hunting, Available online: https:\/\/hodigital.blog.gov.uk\/wp-content\/uploads\/sites\/161\/2020\/03\/Detecting-the-Unknown-A-Guide-to-Threat-Hunting-v2.0.pdf."},{"key":"ref_15","unstructured":"Brown, D. (2024, December 20). Preventing Living Off the Land Attacks. Available online: https:\/\/www.sans.org\/white-papers\/39450\/."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Villal\u00f3n-Huerta, A., Ripoll-Ripoll, I., and Marco-Gisbert, H. (2022). Key requirements for the detection and sharing of behavioral indicators of compromise. Electronics, 11.","DOI":"10.3390\/electronics11030416"},{"key":"ref_17","unstructured":"Vaswani, A. (2017, January 4\u20139). Attention is all you need. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_18","unstructured":"Brown, T.B. (2020). Language models are few-shot learners. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"McCoy, R. (2019). Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference. arXiv.","DOI":"10.18653\/v1\/P19-1334"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bender, E.M., Gebru, T., McMillan-Major, A., and Shmitchell, S. (2021, January 3\u201310). On the dangers of stochastic parrots: Can language models be too big?. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, Virtual Event.","DOI":"10.1145\/3442188.3445922"},{"key":"ref_21","unstructured":"Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J. (2020, January 5\u201310). Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1007\/s11023-020-09548-1","article-title":"GPT-3: Its nature, scope, limits, and consequences","volume":"30","author":"Floridi","year":"2020","journal-title":"Minds Mach."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2053951716679679","DOI":"10.1177\/2053951716679679","article-title":"The ethics of algorithms: Mapping the debate","volume":"3","author":"Mittelstadt","year":"2016","journal-title":"Big Data Soc."},{"key":"ref_24","unstructured":"Microsoft (2024, December 23). Introducing Microsoft 365 Copilot\u2014Your Copilot for Work. Available online: https:\/\/blogs.microsoft.com\/blog\/2023\/03\/16\/introducing-microsoft-365-copilot-your-copilot-for-work\/."},{"key":"ref_25","unstructured":"Crowdstrike (2024, December 23). Introducing Charlotte AI, CrowdStrike\u2019s Generative AI Security Analyst: Ushering in the Future of AI-Powered Cybersecurity. Available online: https:\/\/www.crowdstrike.com\/blog\/crowdstrike-introduces-charlotte-ai-to-deliver-generative-ai-powered-cybersecurity\/."},{"key":"ref_26","unstructured":"Google (2024, December 23). Prompt Engineering for Generative AI. Available online: https:\/\/developers.google.com\/machine-learning\/resources\/prompt-eng."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Reynolds, L., and McDonell, K. (2021, January 8\u201313). Prompt programming for large language models: Beyond the few-shot paradigm. Proceedings of the Extended abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan.","DOI":"10.1145\/3411763.3451760"},{"key":"ref_28","unstructured":"Chen, B., Zhang, Z., Langren\u00e9, N., and Zhu, S. (2023). Unleashing the potential of prompt engineering in Large Language Models: A comprehensive review. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Subramanian, K. (2020). Introducing the Splunk platform. Practical Splunk Search Processing Language: A Guide for Mastering SPL Commands for Maximum Efficiency and Outcome, Apress.","DOI":"10.1007\/978-1-4842-6276-4"},{"key":"ref_30","unstructured":"Sindiramutty, S.R. (2023). Autonomous Threat Hunting: A Future Paradigm for AI-Driven Threat Intelligence. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Liu, X., Tan, Y., Xiao, Z., Zhuge, J., and Zhou, R. (2023, January 9\u201314). Not the end of story: An evaluation of ChatGPT-driven vulnerability description mappings. Proceedings of the Findings of the Association for Computational Linguistics: ACL 2023, Toronto, ON, Canada.","DOI":"10.18653\/v1\/2023.findings-acl.229"},{"key":"ref_32","unstructured":"Moskal, S., Laney, S., Hemberg, E., and O\u2019Reilly, U.M. (2023). Llms killed the script kiddie: How agents supported by large language models change the landscape of network threat testing. arXiv."},{"key":"ref_33","unstructured":"Schwartz, Y., Benshimol, L., Mimran, D., Elovici, Y., and Shabtai, A. (2024). Llmcloudhunter: Harnessing llms for automated extraction of detection rules from cloud-based cti. arXiv."},{"key":"ref_34","unstructured":"LOLBAS-Project (2024, December 20). LOLBAS-Project on GitHub. Available online: https:\/\/github.com\/LOLBAS-Project\/LOLBAS."},{"key":"ref_35","unstructured":"Red Canary (2024, December 20). Atomic Red Team. Available online: https:\/\/github.com\/redcanaryco\/atomic-red-team."},{"key":"ref_36","unstructured":"MITRE (2024, December 20). MITRE ATT&CK Framework. Available online: https:\/\/attack.mitre.org\/."},{"key":"ref_37","unstructured":"Team, G., Anil, R., Borgeaud, S., Alayrac, J.B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A.M., Hauth, A., and Millican, K. (2023). Gemini: A family of highly capable multimodal models. arXiv."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/2\/31\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:06:10Z","timestamp":1760029570000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/2\/31"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,30]]},"references-count":37,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["make7020031"],"URL":"https:\/\/doi.org\/10.3390\/make7020031","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,30]]}}}