{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,1,7]],"date-time":"2025-01-07T05:05:01Z","timestamp":1736226301472,"version":"3.32.0"},"reference-count":58,"publisher":"Oxford University Press (OUP)","issue":"12","license":[{"start":{"date-parts":[[2024,9,29]],"date-time":"2024-09-29T00:00:00Z","timestamp":1727568000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,12,20]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Open-source information platforms such as Twitter continuously provide the latest threat intelligence, including new vulnerabilities and in-the-wild exploitations of advanced persistent threat (APT) groups. Automated extraction of threat intelligence from Twitter has become crucial for defenders to access up-to-date threat knowledge. However, existing studies mainly rely on supervised learning methods to extract threat intelligence knowledge, such as entities, which require a large amount of annotated data. This paper presents Threat Intelligence Mining and Analysis based on Prompt Learning (P-TIMA), a framework specifically crafted for extracting and analyzing threat intelligence from Twitter. P-TIMA employs our innovative few-shot entity recognition method, SecEntPrompt (SEP), built on prompt learning, to extract vulnerability intelligence from Twitter. Additionally, P-TIMA analyzes and profiles the overarching vulnerability intelligence obtained from Twitter, along with in-the-wild exploitation intelligence of APT groups. The SEP improves the average entity recognition F1 score by 3.62-4.40 compared with the best-performing comparison model and outperforms the method based on the large language model on recognition performance and inference time. To validate our framework, we apply P-TIMA to extract vulnerability-related threat intelligence from real Twitter data. Through case studies, we then analyze trends in vulnerability threats and the exploitation capabilities of APT groups. In conclusion, our framework provides a more efficient and accurate method for extracting threat intelligence from Twitter, enabling defenders to stay up-to-date with the latest threat trends and helping them improve their defense strategies against cyber attacks.<\/jats:p>","DOI":"10.1093\/comjnl\/bxae084","type":"journal-article","created":{"date-parts":[[2024,9,30]],"date-time":"2024-09-30T09:52:03Z","timestamp":1727689923000},"page":"3221-3238","source":"Crossref","is-referenced-by-count":0,"title":["P-TIMA: a framework of T witter threat intelligence mining and analysis based on a prompt-learning NER model"],"prefix":"10.1093","volume":"67","author":[{"given":"Yizhe","family":"You","sequence":"first","affiliation":[{"name":"Institute of Information Engineering , Chinese Academy of Sciences, No. 19 Shucun Rd, Haidian District, Beijing 100085,","place":["China"]},{"name":"School of Cyber Security, University of Chinese Academy of Sciences , No. 1 Yanqihu East Rd, Huairou District, Beijing 101408,","place":["China"]}]},{"given":"Zhengwei","family":"Jiang","sequence":"additional","affiliation":[{"name":"Institute of Information Engineering , Chinese Academy of Sciences, No. 19 Shucun Rd, Haidian District, Beijing 100085,","place":["China"]},{"name":"School of Cyber Security, University of Chinese Academy of Sciences , No. 1 Yanqihu East Rd, Huairou District, Beijing 101408,","place":["China"]}]},{"given":"Peian","family":"Yang","sequence":"additional","affiliation":[{"name":"Institute of Information Engineering , Chinese Academy of Sciences, No. 19 Shucun Rd, Haidian District, Beijing 100085,","place":["China"]}]},{"given":"Jun","family":"Jiang","sequence":"additional","affiliation":[{"name":"Institute of Information Engineering , Chinese Academy of Sciences, No. 19 Shucun Rd, Haidian District, Beijing 100085,","place":["China"]},{"name":"School of Cyber Security, University of Chinese Academy of Sciences , No. 1 Yanqihu East Rd, Huairou District, Beijing 101408,","place":["China"]}]},{"given":"Kai","family":"Zhang","sequence":"additional","affiliation":[{"name":"Institute of Information Engineering , Chinese Academy of Sciences, No. 19 Shucun Rd, Haidian District, Beijing 100085,","place":["China"]}]},{"given":"Xuren","family":"Wang","sequence":"additional","affiliation":[{"name":"Information Engineering College , Capital Normal University, 105 West Third Ring Road North, Haidian District, Beijing 100048,","place":["China"]}]},{"given":"Chenpeng","family":"Tu","sequence":"additional","affiliation":[{"name":"Institute of Information Engineering , Chinese Academy of Sciences, No. 19 Shucun Rd, Haidian District, Beijing 100085,","place":["China"]},{"name":"School of Cyber Security , University of Chinese Academy of Sciences, No. 1 Yanqihu East Rd, Huairou District, Beijing 101408,","place":["China"]}]},{"given":"Huamin","family":"Feng","sequence":"additional","affiliation":[{"name":"Beijing Electronic Science and Technology Institute , No. 7 Fufeng Road, Fengtai District, Beijing 100070,","place":["China"]}]}],"member":"286","published-online":{"date-parts":[[2024,9,29]]},"reference":[{"key":"2025010523430593100_ref1","first-page":"433","article-title":"A different cup of TI? The added value of commercial threat intelligence","volume-title":"29th USENIX Security Symposium (USENIX security 20), MA, USA, August","author":"Bouwman"},{"key":"2025010523430593100_ref2","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1007\/978-3-030-58951-6_11","article-title":"Follow the blue bird: a study on threat data published on twitter","volume-title":"Computer Security\u2013ESORICS 2020: 25th European Symposium on Research in Computer Security, ESORICS 2020, Guildford, UK, September","author":"Alves"},{"author":"Kapko","key":"2025010523430593100_ref3","article-title":"Energy providers hit by North Korea-linked lazarus exploiting log4j vmware vulnerabilities"},{"key":"2025010523430593100_ref4"},{"key":"2025010523430593100_ref5"},{"key":"2025010523430593100_ref6"},{"author":"Samani","key":"2025010523430593100_ref7","article-title":"Apt 37 in-the-wild exploitation tweets"},{"key":"2025010523430593100_ref8","first-page":"92","article-title":"# twiti: social listening for threat intelligence","volume-title":"Proceedings of the Web Conference 2021, New York, NY, USA, April, 2021","author":"Shin"},{"key":"2025010523430593100_ref9","first-page":"1","article-title":"Cyberthreat detection from twitter using deep neural networks","volume-title":"2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary, July, 2019","author":"Dion\u00edsio"},{"key":"2025010523430593100_ref10","first-page":"1","article-title":"Towards end-to-end cyberthreat detection from twitter using multi-task learning","volume-title":"2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK, July, 2020","author":"Dion\u00edsio"},{"key":"2025010523430593100_ref11","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1007\/978-3-030-87839-9_3","article-title":"Few-sample named entity recognition for security vulnerability reports by fine-tuning pre-trained language models","volume-title":"International Workshop on Deployable Machine learning for Security Defense, Virtual, august","author":"Yang","year":"2021"},{"key":"2025010523430593100_ref12","first-page":"1188","article-title":"Distributed representations of sentences and documents","volume-title":"International Conference on Machine learning, Beijing, China, June, 2014","author":"Le"},{"key":"2025010523430593100_ref13","first-page":"1532","article-title":"Glove: global vectors for word representation","volume-title":"Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Doha, Qatar, October","author":"Pennington"},{"key":"2025010523430593100_ref14","article-title":"Lstm recurrent neural networks for cybersecurity named entity recognition","volume":"11","author":"Gasmi","year":"2018","journal-title":"ICSEA"},{"key":"2025010523430593100_ref15","first-page":"1","article-title":"Cold-start cybersecurity ontology population using information extraction with lstm","volume-title":"2019 International Conference on Cyber Security for Emerging Technologies (CSET), Doha, Qatar, October, 2019","author":"Gasmi"},{"key":"2025010523430593100_ref16","doi-asserted-by":"crossref","first-page":"259","DOI":"10.26599\/TST.2019.9010033","article-title":"Cybersecurity named entity recognition using bidirectional long short-term memory with conditional random fields","volume":"26","author":"Ma","journal-title":"Tsinghua Science and Technology,"},{"key":"2025010523430593100_ref17","first-page":"1370","article-title":"An effective approach of named entity recognition for cyber threat intelligence","volume-title":"2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Chongqing, China, June, 2020","author":"Wu"},{"author":"Binyamini","key":"2025010523430593100_ref18","article-title":"An automated, end-to-end framework for modeling attacks from vulnerability descriptions"},{"key":"2025010523430593100_ref19","first-page":"29","article-title":"Automatic part-of-speech tagging for security vulnerability descriptions","volume-title":"2021 IEEE\/ACM 18th international conference on mining software repositories (MSR), Madrid, Spain, May, 2021","author":"Yitagesu"},{"key":"2025010523430593100_ref20","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1007\/978-981-15-4828-4_14","article-title":"Deep learning approach for intelligent named entity recognition of cyber security","volume-title":"Advances in Signal Processing and Intelligent Recognition Systems: 5th International Symposium, SIRS 2019, Trivandrum, India, December 18\u201321, 2019, Revised Selected Papers 5, Trivandrum, India, May, 2019","author":"Simran"},{"key":"2025010523430593100_ref21","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4842-2766-4_7","article-title":"Introduction to Keras","volume-title":"Deep Learning with Python: A Hands-on Introduction","author":"Ketkar","year":"2017"},{"key":"2025010523430593100_ref22","doi-asserted-by":"publisher","first-page":"1215","DOI":"10.1093\/comjnl\/bxaa141","article-title":"Cybereyes: Cybersecurity entity recognition model based on graph convolutional network","volume":"64","author":"Fang","year":"2021","journal-title":"The Computer Journal"},{"key":"2025010523430593100_ref23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2019\/6417407","article-title":"Multifeature named entity recognition in information security based on adversarial learning","volume":"2019","author":"Zhang","year":"2019","journal-title":"Security and communication Networks"},{"volume-title":"Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding","author":"Devlin","key":"2025010523430593100_ref24"},{"key":"2025010523430593100_ref25","first-page":"236","article-title":"Joint bert model based cybersecurity named entity recognition","volume-title":"2021 the 4th international conference on software engineering and information management, New York, NY, United States, January","author":"Chen"},{"key":"2025010523430593100_ref26","first-page":"508","article-title":"Named entity recognition method in network security domain based on bert-bilstm-crf","volume-title":"2021 IEEE 21st International Conference on Communication Technology (ICCT), Tianjin, China, October, 2021","author":"He"},{"key":"2025010523430593100_ref27","first-page":"348","article-title":"Named entity recognition in cyber threat intelligence using transformer-based models","volume-title":"2021 IEEE International Conference on Cyber Security and Resilience (CSR), Rhodes, Greece, July, 2021","author":"Evangelatos"},{"key":"2025010523430593100_ref28","first-page":"155","article-title":"Automated corpus annotation for cybersecurity named entity recognition with small keyword dictionary","volume-title":"Intelligent systems and applications: proceedings of the 2021 intelligent systems conference (IntelliSys) volume 3, virtual, September, 2021","author":"Kashihara"},{"key":"2025010523430593100_ref29","first-page":"16","article-title":"Using bert and augmentation in named entity recognition for cybersecurity domain","volume-title":"Natural language processing and information systems: 25th international conference on applications of natural language to information systems, NLDB 2020, Saarbr\u00fccken, Germany, June 24\u201326, 2020, proceedings 25, Saarbr\u00fccken, Germany, June","author":"Tikhomirov"},{"volume-title":"Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language","author":"Kuratov","key":"2025010523430593100_ref30"},{"key":"2025010523430593100_ref31","first-page":"2596","article-title":"A comparative study of deep learning based named entity recognition algorithms for cybersecurity","volume-title":"2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, December, 2020","author":"Dasgupta"},{"key":"2025010523430593100_ref32","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1007\/978-3-030-60450-9_13","article-title":"Ner in threat intelligence domain with tsfl","volume-title":"CCF International Conference on Natural Language Processing and Chinese Computing, Zhengzhou, China, October, 2020","author":"Wang"},{"key":"2025010523430593100_ref33","first-page":"1","article-title":"Collecting indicators of compromise from unstructured text of cybersecurity articles using neural-based sequence labelling","volume-title":"2019 international joint conference on neural networks (IJCNN), Budapest, Hungary, July, 2019","author":"Long"},{"key":"2025010523430593100_ref34","first-page":"86","article-title":"Ransomware entities classification with supervised learning for informal text","volume-title":"2019 International Conference on Cybersecurity (ICoCSec), Negeri Sembilan, Malaysia, September, 2019","author":"Ariffini"},{"key":"2025010523430593100_ref35","doi-asserted-by":"publisher","first-page":"102784","DOI":"10.1016\/j.jisa.2021.102784","article-title":"Nedetector: Automatically extracting cybersecurity neologisms from hacker forums","volume":"58","author":"Li","year":"2021","journal-title":"J. Inf. Secur. Appl."},{"article-title":"Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing","author":"Liu","key":"2025010523430593100_ref36","doi-asserted-by":"crossref","DOI":"10.1145\/3560815"},{"volume-title":"Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference","author":"Schick","key":"2025010523430593100_ref37","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2021.eacl-main.20"},{"author":"Alperin","key":"2025010523430593100_ref38","article-title":"A framework for unsupervised classificiation and data mining of tweets about cyber vulnerabilities"},{"key":"2025010523430593100_ref39","first-page":"272","article-title":"Ti-prompt: Towards a prompt tuning method for few-shot threat intelligence twitter classification","volume-title":"2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC), Los Alamitos, CA, USA, June, 2022","author":"You"},{"volume-title":"Template-Based Named Entity Recognition Using Bart","author":"Cui","key":"2025010523430593100_ref40","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2021.findings-acl.161"},{"volume-title":"Template-Free Prompt Tuning for Few-Shot Ner","author":"Ma","key":"2025010523430593100_ref41","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2022.naacl-main.420"},{"key":"2025010523430593100_ref42","first-page":"3143","article-title":"Using twitter to predict when vulnerabilities will be exploited","volume-title":"Proceedings of the 25th ACM SIGKDD international conference on Knowledge Discovery & Data Mining, Anchorage AK USA, august, 2019","author":"Chen"},{"key":"2025010523430593100_ref43","doi-asserted-by":"publisher","first-page":"1359","DOI":"10.1109\/TSE.2022.3176674","article-title":"Software updates strategies: A quantitative evaluation against advanced persistent threats","volume":"49","author":"Di Tizio","year":"2023","journal-title":"IEEE Transactions on Software Engineering"},{"key":"2025010523430593100_ref44","first-page":"5695","article-title":"Cskg4apt: A cybersecurity knowledge graph for advanced persistent threat organization attribution","volume":"35","author":"Ren","year":"2023","journal-title":"IEEE Trans. Knowl. Data Eng."},{"author":"Elasticsearch","key":"2025010523430593100_ref45","article-title":"Elasticsearch"},{"author":"MITRE","key":"2025010523430593100_ref46","article-title":"Cve records definition."},{"volume-title":"Knowledgeable Prompt-Tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification","author":"Hu","key":"2025010523430593100_ref47"},{"author":"ThaiCERT","key":"2025010523430593100_ref48","article-title":"Threat group cards: a threat actor encyclopedia"},{"key":"2025010523430593100_ref49","first-page":"869","article-title":"Towards the detection of inconsistencies in public security vulnerability reports","volume-title":"28th USENIX security symposium (USENIX security 19), Santa Clara, CA, august, 2019","author":"Dong"},{"author":"OpenAI","key":"2025010523430593100_ref50","article-title":"Introducing chatgpt"},{"author":"Touvron","key":"2025010523430593100_ref51","article-title":"Llama 2: Open foundation and fine-tuned chat models"},{"key":"2025010523430593100_ref52","first-page":"98","article-title":"Deepke: A deep learning based knowledge extraction toolkit for knowledge base population","volume-title":"EMNLP (demos), Abu Dhabi, December","author":"Zhang"},{"key":"2025010523430593100_ref53","doi-asserted-by":"publisher","first-page":"107178","DOI":"10.1016\/j.infsof.2023.107178","article-title":"Automated event extraction of cve descriptions","volume":"158","author":"Wei","year":"2023","journal-title":"Inform Software Technol"},{"author":"Microsoft","key":"2025010523430593100_ref54","article-title":"Lifecycle faq - extended security updates"},{"author":"Wikipedia","key":"2025010523430593100_ref55","article-title":"Patch tuesday"},{"author":"cyberkendra","key":"2025010523430593100_ref56","article-title":"Poc for log4j vulnerability"},{"author":"Tao","key":"2025010523430593100_ref57","article-title":"Another apache log4j vulnerability is actively exploited in the wild (cve-2021-44228) (updated)"},{"author":"\u00d6zkan","key":"2025010523430593100_ref58","article-title":"Cvedetails website"}],"container-title":["The Computer Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/67\/12\/3221\/59428548\/bxae084.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/67\/12\/3221\/59428548\/bxae084.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,6]],"date-time":"2025-01-06T04:31:48Z","timestamp":1736137908000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/comjnl\/article\/67\/12\/3221\/7791002"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,29]]},"references-count":58,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,9,29]]},"published-print":{"date-parts":[[2024,12,20]]}},"URL":"https:\/\/doi.org\/10.1093\/comjnl\/bxae084","relation":{},"ISSN":["0010-4620","1460-2067"],"issn-type":[{"type":"print","value":"0010-4620"},{"type":"electronic","value":"1460-2067"}],"subject":[],"published-other":{"date-parts":[[2024,12]]},"published":{"date-parts":[[2024,9,29]]}}}