{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T16:40:11Z","timestamp":1786120811891,"version":"build-2736575974"},"publisher-location":"New York, NY, USA","reference-count":49,"publisher":"ACM","license":[{"start":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T00:00:00Z","timestamp":1786147200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"name":"National Science Foundation","award":["2442171"],"award-info":[{"award-number":["2442171"]}]},{"name":"Google","award":["Google Academic Research Award (GARA)"],"award-info":[{"award-number":["Google Academic Research Award (GARA)"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,8,9]]},"DOI":"10.1145\/3770855.3817527","type":"proceedings-article","created":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T17:15:37Z","timestamp":1786036537000},"page":"8741-8752","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["CTIConnect: A Benchmark for Retrieval-Augmented LLMs over Heterogeneous Cyber Threat Intelligence"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-8641-8079","authenticated-orcid":false,"given":"Yutong","family":"Cheng","sequence":"first","affiliation":[{"name":"Virginia Tech, Blacksburg, VA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-6607-861X","authenticated-orcid":false,"given":"Yang","family":"Liu","sequence":"additional","affiliation":[{"name":"Virginia Tech, Blacksburg, VA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-5024-3009","authenticated-orcid":false,"given":"Changze","family":"Li","sequence":"additional","affiliation":[{"name":"Virginia Tech, Blacksburg, VA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9745-6802","authenticated-orcid":false,"given":"Dawn","family":"Song","sequence":"additional","affiliation":[{"name":"University of California, Berkeley, Berkeley, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9342-9863","authenticated-orcid":false,"given":"Peng","family":"Gao","sequence":"additional","affiliation":[{"name":"Virginia Tech, Blacksburg, VA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,8,8]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"Md Tanvirul Alam Dipkamal Bhusal Youngja Park and Nidhi Rastogi. 2023. Looking Beyond IoCs: Automatically Extracting Attack Patterns from External CTI. In RAID."},{"key":"e_1_3_2_1_2_1","unstructured":"Md Tanvirul Alam Le Nguyen Dipkamal Bhusal and Nidhi Rastogi. 2025. CTIBench: A Benchmark for Evaluating LLMs in Cyber Threat Intelligence. In NeurIPS."},{"key":"e_1_3_2_1_3_1","unstructured":"Marvin B\u00fcchel Tommaso Paladini Stefano Longari Michele Carminati Stefano Zanero Hodaya Binyamini Gal Engelberg Dan Klein Marco Caselli Andrea Continella Maarten van Steen Andreas Peter Giancarlo Guizzardi and Thijs van Ede. 2025. SoK: Automated TTP Extraction from CTI Reports -- Are We There Yet?. In USENIX Security."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"crossref","unstructured":"Jiawei Chen Hongyu Lin Xianpei Han and Le Sun. 2024. Benchmarking Large Language Models in Retrieval-Augmented Generation. In AAAI.","DOI":"10.1609\/aaai.v38i16.29728"},{"key":"e_1_3_2_1_5_1","volume-title":"Dawn Song, and Peng Gao.","author":"Cheng Yutong","year":"2025","unstructured":"Yutong Cheng, Osama Bajaber, Saimon Amanuel Tsegai, Dawn Song, and Peng Gao. 2025. CTINexus: Automatic Cyber Threat Intelligence Knowledge Graph Construction Using Large Language Models. In EuroS&P."},{"key":"e_1_3_2_1_6_1","volume-title":"Qian Cui, Wei Ding, and Peng Gao.","author":"Cheng Yutong","year":"2026","unstructured":"Yutong Cheng, Changze Li, Raihan Sultan Pasha Basuki, Qian Cui, Wei Ding, and Peng Gao. 2026. TTPrint: Evidence-Grounded TTP Extraction via Diverge-then-Converge Verification. arXiv preprint arXiv:2605.25836 (2026)."},{"key":"e_1_3_2_1_7_1","unstructured":"CrowdStrike. 2025. Threat Intelligence & Hunting. https:\/\/www.crowdstrike.com\/en-us\/platform\/threat-intelligence\/."},{"key":"e_1_3_2_1_8_1","unstructured":"CVE Program. 2025. Common Vulnerabilities and Exposures (CVE). https:\/\/www.cve.org\/."},{"key":"e_1_3_2_1_9_1","unstructured":"Gelei Deng Yi Liu V\u00edctor Mayoral-Vilches Peng Liu Yuekang Li Yuan Xu Tianwei Zhang Yang Liu Martin Pinzger and Stefan Rass. 2024. PentestGPT: Evaluating and Harnessing Large Language Models for Automated Penetration Testing. In USENIX Security."},{"key":"e_1_3_2_1_10_1","unstructured":"Xiaohu Du Ming Wen Jiahao Zhu Zifan Xie Bin Ji Huijun Liu Xuanhua Shi and Hai Jin. 2024. Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning. In ACL Findings."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"crossref","unstructured":"Peng Gao Xiaoyuan Liu Edward Choi Sibo Ma Xinyu Yang and Dawn Song. 2024. ThreatKG: An AI-Powered System for Automated Open-Source Cyber Threat Intelligence Gathering and Management. In LAMPS.","DOI":"10.1145\/3689217.3690613"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"crossref","unstructured":"Peng Gao Fei Shao Xiaoyuan Liu Xusheng Xiao Zheng Qin Fengyuan Xu Prateek Mittal Sanjeev R Kulkarni and Dawn Song. 2021. Enabling Efficient Cyber Threat Hunting with Cyber Threat Intelligence. In ICDE.","DOI":"10.1109\/ICDE51399.2021.00024"},{"key":"e_1_3_2_1_13_1","unstructured":"Chengquan Guo Xun Liu Chulin Xie Andy Zhou Yi Zeng Zinan Lin Dawn Song and Bo Li. 2024. RedCode: Risky Code Execution and Generation Benchmark for Code Agents. In NeurIPS."},{"key":"e_1_3_2_1_14_1","volume-title":"CTIKG: LLM-Powered Knowledge Graph Construction from Cyber Threat Intelligence. In COLM.","author":"Huang Liangyi","year":"2024","unstructured":"Liangyi Huang and Xusheng Xiao. 2024. CTIKG: LLM-Powered Knowledge Graph Construction from Cyber Threat Intelligence. In COLM."},{"key":"e_1_3_2_1_15_1","volume-title":"UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-World Document Analysis. In NeurIPS.","author":"Hui Yulong","year":"2024","unstructured":"Yulong Hui, Yao Lu, and Huanchen Zhang. 2024. UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-World Document Analysis. In NeurIPS."},{"key":"e_1_3_2_1_16_1","volume-title":"Benchmarking, Eliciting, and Enhancing Abilities of Large Language Models in Cyber Threat Intelligence. arXiv preprint arXiv:2405.03446","author":"Ji Hangyuan","year":"2024","unstructured":"Hangyuan Ji, Jian Yang, Linzheng Chai, Chaoren Wei, Liqun Yang, Yunlong Duan, Yunli Wang, Tianzhen Sun, Hongcheng Guo, Tongliang Li, Changyu Ren, and Zhoujun Li. 2024. SEvenLLM: Benchmarking, Eliciting, and Enhancing Abilities of Large Language Models in Cyber Threat Intelligence. arXiv preprint arXiv:2405.03446 (2024)."},{"key":"e_1_3_2_1_17_1","volume-title":"SecBench: A Comprehensive Multi-Dimensional Benchmarking Dataset for LLMs in Cybersecurity. arXiv preprint arXiv:2412.20787","author":"Jing Pengfei","year":"2025","unstructured":"Pengfei Jing, Mengyun Tang, Xiaorong Shi, Xing Zheng, Sen Nie, Shi Wu, Yong Yang, and Xiapu Luo. 2025. SecBench: A Comprehensive Multi-Dimensional Benchmarking Dataset for LLMs in Cybersecurity. arXiv preprint arXiv:2412.20787 (2025)."},{"key":"e_1_3_2_1_18_1","unstructured":"Hwiwon Lee Ziqi Zhang Hanxiao Lu and Lingming Zhang. 2025. SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks. In NeurIPS."},{"key":"e_1_3_2_1_19_1","unstructured":"Kuan Li Liwen Zhang Yong Jiang Pengjun Xie Fei Huang Shuai Wang and Minhao Cheng. 2025. LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs -- No Silver Bullet for LC or RAG Routing. In ICML."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"crossref","unstructured":"Xiaojing Liao Kan Yuan XiaoFeng Wang Zhou Li Luyi Xing and Raheem Beyah. 2016. Acing the IOC Game: Toward Automatic Discovery and Analysis of Open-Source Cyber Threat Intelligence. In CCS.","DOI":"10.1145\/2976749.2978315"},{"key":"e_1_3_2_1_21_1","unstructured":"Jiawei Liu Nirav Diwan Zhe Wang Haoyu Zhai Xiaona Zhou Kiet Nguyen Tianjiao Yu Muntasir Wahed Yinlin Deng Hadjer Benkraouda Yuxiang Wei Lingming Zhang Ismini Lourentzou and Gang Wang. 2025a. PurpCode: Reasoning for Safer Code Generation. In NeurIPS."},{"key":"e_1_3_2_1_22_1","volume-title":"LLM-Powered Static Binary Taint Analysis. ACM TOSEM","author":"Liu Puzhuo","year":"2025","unstructured":"Puzhuo Liu, Chengnian Sun, Yaowen Zheng, Xuan Feng, Chuan Qin, Yuncheng Wang, Zhenyang Xu, Zhi Li, Peng Di, Yu Jiang, and Limin Sun. 2025c. LLM-Powered Static Binary Taint Analysis. ACM TOSEM (2025)."},{"key":"e_1_3_2_1_23_1","volume-title":"CyLens: Towards Reinventing Cyber Threat Intelligence in the Paradigm of Agentic Large Language Models. arXiv preprint arXiv:2502.20791","author":"Liu Xiaoqun","year":"2025","unstructured":"Xiaoqun Liu, Jiacheng Liang, Qiben Yan, Jiyong Jang, Sicheng Mao, Muchao Ye, Jinyuan Jia, and Zhaohan Xi. 2025b. CyLens: Towards Reinventing Cyber Threat Intelligence in the Paradigm of Agentic Large Language Models. arXiv preprint arXiv:2502.20791 (2025)."},{"key":"e_1_3_2_1_24_1","volume-title":"Definition: Threat Intelligence. Technical Report. Gartner. https:\/\/www.gartner.com\/en\/documents\/2487216","author":"McMillan Rob","year":"2013","unstructured":"Rob McMillan. 2013. Definition: Threat Intelligence. Technical Report. Gartner. https:\/\/www.gartner.com\/en\/documents\/2487216"},{"key":"e_1_3_2_1_25_1","unstructured":"Yuqiao Meng Luoxi Tang Feiyang Yu Xi Li Guanhua Yan Ping Yang and Zhaohan Xi. 2026. Benchmarking LLM-Assisted Blue Teaming via Standardized Threat Hunting. In ICML."},{"key":"e_1_3_2_1_26_1","unstructured":"Microsoft. 2024. Microsoft Security Copilot. https:\/\/www.microsoft.com\/en-us\/security\/business\/ai-machine-learning\/microsoft-security-copilot."},{"key":"e_1_3_2_1_27_1","unstructured":"MITRE. 2025a. Common Attack Pattern Enumeration and Classification (CAPEC). https:\/\/capec.mitre.org\/."},{"key":"e_1_3_2_1_28_1","unstructured":"MITRE. 2025b. Common Weakness Enumeration (CWE). https:\/\/cwe.mitre.org\/."},{"key":"e_1_3_2_1_29_1","unstructured":"MITRE. 2025c. MITRE ATT&CK. https:\/\/attack.mitre.org\/."},{"key":"e_1_3_2_1_30_1","volume-title":"Mohan Baruwal Chhetri, and Bao Quoc Vo","author":"Nguyen Hoang Cuong","year":"2025","unstructured":"Hoang Cuong Nguyen, Shahroz Tariq, Mohan Baruwal Chhetri, and Bao Quoc Vo. 2025. Towards Effective Identification of Attack Techniques in Cyber Threat Intelligence Reports using Large Language Models. In WWW Companion."},{"key":"e_1_3_2_1_31_1","volume-title":"NeurIPS Workshop.","author":"Nunez Ana","year":"2024","unstructured":"Ana Nunez, Nafis Tanveer Islam, Sumit Jha, and Peyman Najafirad. 2024. AutoSafeCoder: A Multi-Agent Framework for Securing LLM Code Generation through Static Analysis and Fuzz Testing. In NeurIPS Workshop."},{"key":"e_1_3_2_1_32_1","unstructured":"Palo Alto Networks. 2025. Unit 42 -- Threat Intelligence Research. https:\/\/unit42.paloaltonetworks.com\/."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Yuval Schwartz Lavi Benshimol Dudu Mimran Yuval Elovici and Asaf Shabtai. 2025. LLMCloudHunter: Harnessing LLMs for Automated Extraction of Detection Rules from Cloud-Based CTI. In WWW.","DOI":"10.1145\/3696410.3714798"},{"key":"e_1_3_2_1_34_1","volume-title":"Meet Udeshi, Brendan Dolan-Gavitt, Haoran Xi, Kimberly Milner, Boyuan Chen, Max Yin, Siddharth Garg, Prashanth Krishnamurthy, Farshad Khorrami, Ramesh Karri, and Muhammad Shafique.","author":"Shao Minghao","year":"2024","unstructured":"Minghao Shao, Sofija Jancheska, Meet Udeshi, Brendan Dolan-Gavitt, Haoran Xi, Kimberly Milner, Boyuan Chen, Max Yin, Siddharth Garg, Prashanth Krishnamurthy, Farshad Khorrami, Ramesh Karri, and Muhammad Shafique. 2024. NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security. In NeurIPS."},{"key":"e_1_3_2_1_35_1","volume-title":"Time for aCTIon: Automated Analysis of Cyber Threat Intelligence in the Wild. arXiv preprint arXiv:2307.10214","author":"Siracusano Giuseppe","year":"2023","unstructured":"Giuseppe Siracusano, Davide Sanvito, Roberto Gonzalez, Manikantan Srinivasan, Sivakaman Kamatchi, Wataru Takahashi, Masaru Kawakita, Takahiro Kakumaru, and Roberto Bifulco. 2023. Time for aCTIon: Automated Analysis of Cyber Threat Intelligence in the Wild. arXiv preprint arXiv:2307.10214 (2023)."},{"key":"e_1_3_2_1_36_1","volume-title":"Cyber Threat Intelligence Mining for Proactive Cybersecurity Defense: A Survey and New Perspectives","author":"Sun Nan","year":"2023","unstructured":"Nan Sun, Ming Ding, Jiaojiao Jiang, Weikang Xu, Xiaoxing Mo, Yonghang Tai, and Jun Zhang. 2023. Cyber Threat Intelligence Mining for Proactive Cybersecurity Defense: A Survey and New Perspectives. IEEE Communications Surveys & Tutorials (2023)."},{"key":"e_1_3_2_1_37_1","unstructured":"Yixuan Tang and Yi Yang. 2024. MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries. In COLM."},{"key":"e_1_3_2_1_38_1","unstructured":"Trend Micro. 2025. Threat Intelligence Center. https:\/\/www.trendmicro.com\/vinfo\/us\/security\/threat-intelligence-center."},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"crossref","unstructured":"Harsh Trivedi Niranjan Balasubramanian Tushar Khot and Ashish Sabharwal. 2023. Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. In ACL.","DOI":"10.18653\/v1\/2023.acl-long.557"},{"key":"e_1_3_2_1_40_1","volume-title":"Enabling Efficient Attack Investigation via Human-in-the-Loop Security Analysis. PVLDB","author":"Tsegai Saimon Amanuel","year":"2025","unstructured":"Saimon Amanuel Tsegai, Xinyu Yang, Haoyuan Liu, and Peng Gao. 2025. Enabling Efficient Attack Investigation via Human-in-the-Loop Security Analysis. PVLDB (2025)."},{"key":"e_1_3_2_1_41_1","volume-title":"LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks","author":"Ullah Saad","unstructured":"Saad Ullah, Mingji Han, Saurabh Pujar, Hammond Pearce, Ayse Coskun, and Gianluca Stringhini. 2024. LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks. In IEEE S&P."},{"key":"e_1_3_2_1_42_1","unstructured":"Mark Vero Niels M\u00fcndler Victor Chibotaru Veselin Raychev Maximilian Baader Nikola Jovanovi\u0107 Jingxuan He and Martin Vechev. 2025. BaxBench: Can LLMs Generate Correct and Secure Backends?. In ICML."},{"key":"e_1_3_2_1_43_1","volume-title":"Lyu","author":"Wen Xin-Cheng","year":"2024","unstructured":"Xin-Cheng Wen, Cuiyun Gao, Shuzheng Gao, Yang Xiao, and Michael R. Lyu. 2024. SCALE: Constructing Structured Natural Language Comment Trees for Software Vulnerability Detection. In ISSTA."},{"key":"e_1_3_2_1_44_1","unstructured":"Danning Xie Mingwei Zheng Xuwei Liu Jiannan Wang Chengpeng Wang Lin Tan and Xiangyu Zhang. 2025. CoRe: Benchmarking LLMs' Code Reasoning Capabilities through Static Analysis Tasks. In NeurIPS."},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3689736"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"crossref","unstructured":"Xiao Yang Kai Sun Hao Xin Yushi Sun Nikita Bhalla Xiangsen Chen Sajal Choudhary Rongze Daniel Gui Ziran Will Jiang Ziyu Jiang Lingkun Kong Brian Moran Jiaqi Wang Yifan Ethan Xu An Yan Chenyu Yang Eting Yuan Hanwen Zha Nan Tang Lei Chen Nicolas Scheffer Yue Liu Nirav Shah Rakesh Wanga Anuj Kumar Wen tau Yih and Xin Luna Dong. 2024b. CRAG -- Comprehensive RAG Benchmark. In NeurIPS.","DOI":"10.52202\/079017-0335"},{"key":"e_1_3_2_1_47_1","volume-title":"GPTFuzzer: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts. arXiv preprint arXiv:2309.10253","author":"Yu Jiahao","year":"2024","unstructured":"Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. 2024. GPTFuzzer: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts. arXiv preprint arXiv:2309.10253 (2024)."},{"key":"e_1_3_2_1_48_1","volume-title":"Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models. In ICLR.","author":"Zhang Andy K","year":"2025","unstructured":"Andy K Zhang, Neil Perry, Riya Dulepet, Joey Ji, Celeste Menders, Justin W Lin, Eliot Jones, Gashon Hussein, Samantha Liu, Donovan Jasper, et al., 2025. Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models. In ICLR."},{"key":"e_1_3_2_1_49_1","unstructured":"Yuxuan Zhu Antony Kellermann Dylan Bowman Philip Li Akul Gupta Adarsh Danda Richard Fang Conner Jensen Eric Ihli Jason Benn et al. 2025. CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities. In ICML."}],"event":{"name":"KDD '26: The 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining","location":"Jeju Island Republic of Korea","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"]},"container-title":["Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2"],"original-title":[],"deposited":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T01:26:12Z","timestamp":1786065972000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3770855.3817527"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,8]]},"references-count":49,"alternative-id":["10.1145\/3770855.3817527","10.1145\/3770855"],"URL":"https:\/\/doi.org\/10.1145\/3770855.3817527","relation":{},"subject":[],"published":{"date-parts":[[2026,8,8]]},"assertion":[{"value":"2026-08-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}