{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T04:47:52Z","timestamp":1775882872584,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":40,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,11,30]],"date-time":"2023-11-30T00:00:00Z","timestamp":1701302400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,11,30]]},"DOI":"10.1145\/3611643.3613083","type":"proceedings-article","created":{"date-parts":[[2023,11,30]],"date-time":"2023-11-30T23:14:38Z","timestamp":1701386078000},"page":"2082-2086","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":89,"title":["Getting pwn\u2019d by AI: Penetration Testing with Large Language Models"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-2484-0109","authenticated-orcid":false,"given":"Andreas","family":"Happe","sequence":"first","affiliation":[{"name":"TU Wien, Vienna, Austria"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8619-1271","authenticated-orcid":false,"given":"J\u00fcrgen","family":"Cito","sequence":"additional","affiliation":[{"name":"TU Wien, Vienna, Austria"}]}],"member":"320","published-online":{"date-parts":[[2023,11,30]]},"reference":[{"key":"e_1_3_2_2_1_1","unstructured":"AIAAIC. 2023. AIAAIC Repository of incidents and controversies related to AI algorithms and automation.. https:\/\/www.aiaaic.org\/"},{"key":"e_1_3_2_2_2_1","volume-title":"The Wassenaar Arrangement on Export Controls for Conventional Arms and Dual-Use Goods and Technologies. https:\/\/www.wassenaar.org\/","author":"Arrangement The Wassenaar","year":"1982","unstructured":"The Wassenaar Arrangement. 1982. The Wassenaar Arrangement on Export Controls for Conventional Arms and Dual-Use Goods and Technologies. https:\/\/www.wassenaar.org\/"},{"key":"e_1_3_2_2_3_1","unstructured":"MITRE ATT&CK. 2020. Abuse Elevation Control Mechanism: Sudo and Sudo Caching. https:\/\/attack.mitre.org\/techniques\/T1548\/003\/"},{"key":"e_1_3_2_2_4_1","unstructured":"MITRE ATT&CK. 2020. Steal or Forge Kerberos Tickets: Kerberoasting. https:\/\/attack.mitre.org\/techniques\/T1558\/003\/"},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"publisher","unstructured":"Edward Beeching Younes Belkada Kashif Rasul Lewis Tunstall Leandro von Werra Nazneen Rajani and Nathan Lambert. 2023. StackLLaMA: An RL Fine-tuned LLaMA Model for Stack Exchange Question and Answering. https:\/\/doi.org\/10.57967\/hf\/0513 10.57967\/hf\/0513","DOI":"10.57967\/hf"},{"key":"e_1_3_2_2_6_1","volume-title":"Augmented Education in the Global Age","author":"Brynjolfsson Erik","unstructured":"Erik Brynjolfsson. 2023. The turing trap: The promise & peril of human-like artificial intelligence. In Augmented Education in the Global Age. Routledge, 103\u2013116."},{"key":"e_1_3_2_2_7_1","volume-title":"Generative AI at Work. NBER Working Paper No. 31161","author":"Brynjolfsson Erik","unstructured":"Erik Brynjolfsson, Danielle Li, and Lindsey Raymond. 2023. Generative AI at Work. NBER Working Paper No. 31161.. National Bureau of Economic Research, April."},{"key":"e_1_3_2_2_8_1","volume-title":"Cambridge International Workshop on Security Protocols. 55\u201361","author":"Bukac Vit","year":"2014","unstructured":"Vit Bukac, Vaclav Lorenc, and Vashek Maty\u00e1\u0161. 2014. Red queen\u2019s race: APT win-win game. In Cambridge International Workshop on Security Protocols. 55\u201361."},{"key":"e_1_3_2_2_9_1","volume-title":"Free Dolly: Introducing the World\u2019s First Truly Open Instruction-Tuned LLM. https:\/\/www.databricks.com\/blog\/2023\/04\/12\/dolly-first-open-commercially-viable-instruction-tuned-llm","author":"Conover Mike","year":"2023","unstructured":"Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free Dolly: Introducing the World\u2019s First Truly Open Instruction-Tuned LLM. https:\/\/www.databricks.com\/blog\/2023\/04\/12\/dolly-first-open-commercially-viable-instruction-tuned-llm"},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3545945.3569823"},{"key":"e_1_3_2_2_11_1","unstructured":"The Economist. 2022. Huge foundation models are turbo-charging AI progress. https:\/\/www.economist.com\/interactive\/briefing\/2022\/06\/11\/huge-foundation-models-are-turbo-charging-ai-progress"},{"key":"e_1_3_2_2_12_1","unstructured":"The Economist. 2023. Large creative AI models will transform lives and labour markets. https:\/\/www.economist.com\/interactive\/science-and-technology\/2023\/04\/22\/large-creative-ai-models-will-transform-how-we-live-and-work"},{"key":"e_1_3_2_2_13_1","volume-title":"Koala: A Dialogue Model for Academic Research. Blog post. https:\/\/bair.berkeley.edu\/blog\/2023\/04\/03\/koala\/","author":"Geng Xinyang","year":"2023","unstructured":"Xinyang Geng, Arnav Gudibande, Hao Liu, Eric Wallace, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023. Koala: A Dialogue Model for Academic Research. Blog post. https:\/\/bair.berkeley.edu\/blog\/2023\/04\/03\/koala\/"},{"key":"e_1_3_2_2_14_1","unstructured":"Georgi Gerganov. 2023. llama.cpp: Inference of LLaMA model in pure C\/C++. https:\/\/github.com\/ggerganov\/llama.cpp"},{"key":"e_1_3_2_2_15_1","unstructured":"Significant Gravitas. 2023. Auto-GPT: An Autonomous GPT-4 Experiment. https:\/\/github.com\/Significant-Gravitas\/Auto-GPT"},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3613900"},{"key":"e_1_3_2_2_17_1","unstructured":"Richard Harang and Felipe N Ducau. 2018. Measuring the speed of the Red Queen\u2019s Race. BlackHat: Las Vegas NV USA."},{"key":"e_1_3_2_2_18_1","unstructured":"(ISC)2. 2022. (ISC)2 CYBERSECURITY WORKFORCE STUDY 2022. https:\/\/www.isc2.org\/\/-\/media\/ISC2\/Research\/2022-WorkForce-Study\/ISC2-Cybersecurity-Workforce-Study.ashx"},{"key":"e_1_3_2_2_19_1","unstructured":"Sydney Lake. 2022. The cybersecurity industry is short 3.4 million workers\u2014that\u2019s good news for cyber wages. https:\/\/fortune.com\/education\/articles\/the-cybersecurity-industry-is-short-3-4-million-workers-thats-good-news-for-cyber-wages\/"},{"key":"e_1_3_2_2_20_1","volume-title":"Cobalt Strike: Favorite Tool from APT to Crimeware. https:\/\/www.proofpoint.com\/us\/blog\/threat-insight\/cobalt-strike-favorite-tool-apt-crimeware","author":"Larson Selena","year":"2021","unstructured":"Selena Larson and Daniel Blackford. 2021. Cobalt Strike: Favorite Tool from APT to Crimeware. https:\/\/www.proofpoint.com\/us\/blog\/threat-insight\/cobalt-strike-favorite-tool-apt-crimeware"},{"key":"e_1_3_2_2_21_1","unstructured":"lin.security. 2018. Lin.Security: 1. https:\/\/www.vulnhub.com\/entry\/linsecurity-1 244\/"},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501825"},{"key":"e_1_3_2_2_23_1","volume-title":"Vanessa Parli, Yoav Shoham, Russell Wald, Jack Clark, and Raymond Perraul.","author":"Maslej Nestor","year":"2023","unstructured":"Nestor Maslej, Loredana Fattorini, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Helen Ngo, Juan Carlos Niebles, Vanessa Parli, Yoav Shoham, Russell Wald, Jack Clark, and Raymond Perraul. 2023. The AI Index 2023 Annual Report. https:\/\/aiindex.stanford.edu\/wp-content\/uploads\/2023\/04\/HAI_AI-Index-Report_2023.pdf"},{"key":"e_1_3_2_2_24_1","volume-title":"Sam Altman: Size of LLMs won\u2019t matter as much moving forward. https:\/\/techcrunch.com\/2023\/04\/14\/sam-altman-size-of-llms-wont-matter-as-much-moving-forward\/","author":"Miller Ron","year":"2023","unstructured":"Ron Miller. 2023. Sam Altman: Size of LLMs won\u2019t matter as much moving forward. https:\/\/techcrunch.com\/2023\/04\/14\/sam-altman-size-of-llms-wont-matter-as-much-moving-forward\/"},{"key":"e_1_3_2_2_25_1","unstructured":"Yohei Nakajima. 2023. BabyAGI. https:\/\/github.com\/yoheinakajima\/babyagi"},{"key":"e_1_3_2_2_26_1","unstructured":"Yohei Nakajima. 2023. Introducing Task-driven Autonomous Agent. https:\/\/twitter.com\/yoheinakajima\/status\/1640934493489070080"},{"key":"e_1_3_2_2_27_1","unstructured":"Yohei Nakajima. 2023. Task-driven Autonomous Agent Utilizing GPT-4 Pinecone and LangChain for Diverse Applications. https:\/\/yoheinakajima.com\/task-driven-autonomous-agent-utilizing-gpt-4-pinecone-and-langchain-for-diverse-applications\/"},{"key":"e_1_3_2_2_28_1","volume-title":"Percy Liang, and Michael S.","author":"Park Joon Sung","year":"2023","unstructured":"Joon Sung Park, Joseph C. O\u2019Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative Agents: Interactive Simulacra of Human Behavior. arxiv:2304.03442."},{"key":"e_1_3_2_2_29_1","unstructured":"Baolin Peng Michel Galley Pengcheng He Hao Cheng Yujia Xie Yu Hu Qiuyuan Huang Lars Liden Zhou Yu Weizhu Chen and Jianfeng Gao. 2023. Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback. arxiv:2302.12813."},{"key":"e_1_3_2_2_30_1","unstructured":"Carlos Polop. 2023. LinPEAS - Linux Privilege Escalation Awesome Script. https:\/\/github.com\/carlospolop\/PEASS-ng\/tree\/master\/linPEAS"},{"key":"e_1_3_2_2_31_1","unstructured":"Katyanna Quach. 2023. LLaMA drama as Meta\u2019s mega language model leaks. https:\/\/www.theregister.com\/2023\/03\/08\/meta_llama_ai_leak\/"},{"key":"e_1_3_2_2_32_1","volume-title":"Szu Yu Chean, and Nitasha Tiku","author":"Schaul Kevin","year":"2023","unstructured":"Kevin Schaul, Szu Yu Chean, and Nitasha Tiku. 2023. Inside the secret list of websites that make AI like ChatGPT sound smart. https:\/\/www.washingtonpost.com\/technology\/interactive\/2023\/ai-chatbot-learning\/"},{"key":"e_1_3_2_2_33_1","unstructured":"Yongliang Shen Kaitao Song Xu Tan Dongsheng Li Weiming Lu and Yueting Zhuang. 2023. HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace. arxiv:2303.17580."},{"key":"e_1_3_2_2_34_1","unstructured":"Cybereason Global SOC and Incident Response Team. 2023. Sliver C2 Leveraged by Many Threat Actors. https:\/\/www.cybereason.com\/blog\/sliver-c2-leveraged-by-many-threat-actors"},{"key":"e_1_3_2_2_35_1","unstructured":"stability.ai. 2023. Stability AI Launches the First of its StableLM Suite of Language Models. https:\/\/stability.ai\/blog\/stability-ai-launches-the-first-of-its-stablelm-suite-of-language-models"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2022.3209479"},{"key":"e_1_3_2_2_37_1","volume-title":"Technical report","author":"Strom Blake E","unstructured":"Blake E Strom, Andy Applebaum, Doug P Miller, Kathryn C Nickels, Adam G Pennington, and Cody B Thomas. 2018. Mitre att&ck: Design and philosophy. In Technical report. The MITRE Corporation."},{"key":"e_1_3_2_2_38_1","volume-title":"Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus.","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. Emergent Abilities of Large Language Models. arxiv:2206.07682."},{"key":"e_1_3_2_2_39_1","volume-title":"Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199.","author":"Zhang Renrui","year":"2023","unstructured":"Renrui Zhang, Jiaming Han, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, Peng Gao, and Yu Qiao. 2023. Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199."},{"key":"e_1_3_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-022-01653-1"}],"event":{"name":"ESEC\/FSE '23: 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering","location":"San Francisco CA USA","acronym":"ESEC\/FSE '23","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering"]},"container-title":["Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3611643.3613083","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3611643.3613083","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:10Z","timestamp":1750178230000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3611643.3613083"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,30]]},"references-count":40,"alternative-id":["10.1145\/3611643.3613083","10.1145\/3611643"],"URL":"https:\/\/doi.org\/10.1145\/3611643.3613083","relation":{},"subject":[],"published":{"date-parts":[[2023,11,30]]},"assertion":[{"value":"2023-11-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}