{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,5]],"date-time":"2026-04-05T20:34:24Z","timestamp":1775421264936,"version":"3.50.1"},"reference-count":34,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2025,9,22]],"date-time":"2025-09-22T00:00:00Z","timestamp":1758499200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Sichuan Science and Technology Program","award":["2024NSFSC0515"],"award-info":[{"award-number":["2024NSFSC0515"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>Cyberattacks, especially Advanced Persistent Threats (APTs), have become more complex. These evolving threats challenge traditional defense systems, which struggle to counter long-lasting and covert attacks. Cybersecurity Knowledge Graphs (CKGs), enabled through the integration of multi-source CTI, introduce novel approaches for proactive defense. However, building CKGs faces challenges such as unclear terminology, overlapping entity relationships in attack chains, and differences in CTI across sources. To tackle these challenges, we propose the CyberKG framework, which improves entity recognition and relation extraction using a SecureBERT_Plus-BiLSTM-Attention-CRF joint architecture. Semantic features are captured using a domain-adapted SecureBERT_Plus model, while temporal dependencies are modeled through BiLSTM. Attention mechanisms highlight key cross-sentence relationships, while CRF incorporates ATT&amp;CK rule constraints. Hierarchical clustering (HAC), based on contextual embeddings, facilitates dynamic entity disambiguation and semantic fusion. Experimental evaluations on the DNRTI and MalwareDB datasets demonstrate strong performance in extraction accuracy, entity normalization, and the resolution of overlapping relations. The constructed knowledge graph supports APT tracking, attack-chain provenance, proactive defense prediction.<\/jats:p>","DOI":"10.3390\/informatics12030100","type":"journal-article","created":{"date-parts":[[2025,9,22]],"date-time":"2025-09-22T12:08:03Z","timestamp":1758542883000},"page":"100","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["CyberKG: Constructing a Cybersecurity Knowledge Graph Based on SecureBERT_Plus for CTI Reports"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3615-1129","authenticated-orcid":false,"given":"Binyong","family":"Li","sequence":"first","affiliation":[{"name":"School of Cybersecurity (Xin Gu Industrial College), Chengdu University of Information Technology, Chengdu 610225, China"},{"name":"Advanced Cryptography and System Security Key Laboratory of Sichuan Province, Chengdu 610225, China"},{"name":"SUGON Industrial Control and Security Center, Chengdu 610225, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-2326-2594","authenticated-orcid":false,"given":"Qiaoxi","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Cybersecurity (Xin Gu Industrial College), Chengdu University of Information Technology, Chengdu 610225, China"}]},{"given":"Chuang","family":"Deng","sequence":"additional","affiliation":[{"name":"Center for Power Emergency Management, State Grid Sichuan Electric Power Corporation, Chengdu 610041, China"}]},{"given":"Hua","family":"Pan","sequence":"additional","affiliation":[{"name":"Guangxi Beitou Innovation Technology Investment Group Co., Ltd., Nanning 530029, China"}]}],"member":"1968","published-online":{"date-parts":[[2025,9,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Yan, Z., and Liu, J. (2020, January 27\u201329). A Review on Application of Knowledge Graph in Cybersecurity. Proceedings of the 2020 International Signal Processing, Communications and Engineering Management Conference (ISPCEM), Montreal, QC, Canada.","DOI":"10.1109\/ISPCEM52197.2020.00055"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/j.neucom.2016.12.075","article-title":"Joint Entity and Relation Extraction Based on a Hybrid Neural Network","volume":"257","author":"Zheng","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2350274","DOI":"10.1142\/S0218126623502742","article-title":"PipCKG-BS: A Method to Build Cybersecurity Knowledge Graph for Blockchain Systems via the Pipeline Approach","volume":"32","author":"Li","year":"2023","journal-title":"J. Circuits Syst. Comput."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"121448","DOI":"10.1016\/j.eswa.2023.121448","article-title":"A Method for Constructing a Machining Knowledge Graph Using an Improved Transformer","volume":"237","author":"Guo","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Niakanlahiji, A., Wei, J., and Chu, B.T. (2018, January 10\u201313). A Natural Language Processing Based Trend Analysis of Advanced Persistent Threat Techniques. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA.","DOI":"10.1109\/BigData.2018.8622255"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Satvat, K., Gjomemo, R., and Venkatakrishnan, V. (2021, January 6\u201310). Extractor: Extracting Attack Behavior from Threat Reports. Proceedings of the 2021 IEEE European Symposium on Security and Privacy (EuroS&P), Vienna, Austria.","DOI":"10.1109\/EuroSP51992.2021.00046"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"107524","DOI":"10.1016\/j.knosys.2021.107524","article-title":"Open-CyKG: An Open Cyber Threat Intelligence Knowledge Graph","volume":"233","author":"Sarhan","year":"2021","journal-title":"Knowl.-Based Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2341","DOI":"10.1007\/s13042-020-01122-6","article-title":"Automatic Extraction of Named Entities of Cyber Threats Using a Deep Bi-LSTM-CRF Network","volume":"11","author":"Kim","year":"2020","journal-title":"Int. J. Mach. Learn. Cybern."},{"key":"ref_9","first-page":"5695","article-title":"CSKG4APT: A Cybersecurity Knowledge Graph for Advanced Persistent Threat Organization Attribution","volume":"35","author":"Ren","year":"2022","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_10","first-page":"9875199","article-title":"CTI View: APT Threat Intelligence Analysis System","volume":"2022","author":"Zhou","year":"2022","journal-title":"Secur. Commun. Netw."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1016\/j.procs.2023.01.027","article-title":"Study of Word Embeddings for Enhanced Cyber Security Named Entity Recognition","volume":"218","author":"Srivastava","year":"2023","journal-title":"Procedia Comput. Sci."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, X., He, S., Xiong, Z., Wei, X., Jiang, Z., Chen, S., and Jiang, J. (2022, January 4\u20136). APTNER: A Specific Dataset for NER Missions in Cyber Threat Intelligence Field. Proceedings of the 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD), Hangzhou, China.","DOI":"10.1109\/CSCWD54268.2022.9776031"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Bekoulis, G., Deleu, J., Demeester, T., and Develder, C. (2018). Adversarial Training for Multi-Context Joint Entity and Relation Extraction. arXiv.","DOI":"10.18653\/v1\/D18-1307"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Sind, X., Liu, T., Fang, Z., and Li, Q. (2020, January 19\u201324). Joint Entity Linking and Relation Extraction with Neural Networks for Knowledge Base Population. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.","DOI":"10.1109\/IJCNN48605.2020.9207021"},{"key":"ref_15","unstructured":"Zhao, J., Yan, Q., Liu, X., Li, B., and Zuo, G. (2020, January 14\u201316). Cyber Threat Intelligence Modeling Based on Heterogeneous Graph Convolutional Network. Proceedings of the 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020), Taipei, China."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"103371","DOI":"10.1016\/j.cose.2023.103371","article-title":"A Framework for Threat Intelligence Extraction and Fusion","volume":"132","author":"Guo","year":"2023","journal-title":"Comput. Secur."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wang, X., Xiong, M., Luo, Y., Li, N., Jiang, Z., and Xiong, Z. (2020, January 29). Joint Learning for Document-Level Threat Intelligence Relation Extraction and Coreference Resolution Based on GCN. Proceedings of the 2020 IEEE 19th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Guangzhou, China.","DOI":"10.1109\/TrustCom50675.2020.00083"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zuo, J., Gao, Y., Li, X., and Yuan, J. (2022, January 4). An End-to-End Entity and Relation Joint Extraction Model for Cyber Threat Intelligence. Proceedings of the 2022 7th International Conference on Big Data Analytics (ICBDA), Guangzhou, China.","DOI":"10.1109\/ICBDA55095.2022.9760342"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"103579","DOI":"10.1016\/j.cose.2023.103579","article-title":"CyberEntRel: Joint Extraction of Cyber Entities and Relations Using Deep Learning","volume":"136","author":"Ahmed","year":"2024","journal-title":"Comput. Secur."},{"key":"ref_20","unstructured":"Dey, R., Debnath, A., Dutta, S.K., Ghosh, K., Mitra, A., Roychowdhury, A., and Sen, J. (2024). Semantic Stealth: Adversarial Text Attacks on NLP Using Several Methods. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1145\/3652594","article-title":"CySecBERT: A Domain-Adapted Language Model for the Cybersecurity Domain","volume":"27","author":"Bayer","year":"2024","journal-title":"ACM Trans. Priv. Secur."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Park, Y., and You, W. (2023, January 6\u201310). A Pretrained Language Model for Cyber Threat Intelligence. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, Singapore.","DOI":"10.18653\/v1\/2023.emnlp-industry.12"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1007\/978-3-031-25538-0_3","article-title":"SecureBERT: A Domain-Specific Language Model for Cybersecurity","volume":"Volume 462","author":"Li","year":"2023","journal-title":"Proceedings of the Security and Privacy in Communication Networks"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"105460","DOI":"10.1016\/j.engappai.2022.105460","article-title":"UD_BBC: Named Entity Recognition in Social Network Combined BERT-BiLSTM-CRF with Active Learning","volume":"116","author":"Li","year":"2022","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Xu, H., Fan, G., Kuang, G., and Wang, C. (2023). Exploring the Potential of BERT-BiLSTM-CRF and the Attention Mechanism in Building a Tourism Knowledge Graph. Electronics, 12.","DOI":"10.3390\/electronics12041010"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1109\/TCBB.2019.2939128","article-title":"GrantExtractor: Accurate Grant Support Information Extraction from Biomedical Fulltext Based on Bi-LSTM-CRF","volume":"18","author":"Dai","year":"2021","journal-title":"IEEE\/ACM Trans. Comput. Biol. Bioinform."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Xu, Y., Tan, X., Tong, X., and Zhang, W. (2024). Robust Chinese Named Entity Recognition Method Based on Integrating Dual-Layer Features and CSBERT. Appl. Sci., 14.","DOI":"10.3390\/app14031060"},{"key":"ref_28","unstructured":"Syed, Z., Padia, A., Finin, T., Mathews, L., and Joshi, A. (2016, January 12). UCO: A Unified Cybersecurity Ontology. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), Palo Alto, CA, USA."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"104120","DOI":"10.1016\/j.cose.2024.104120","article-title":"Entity and Relation Extractions for Threat Intelligence Knowledge Graphs","volume":"148","author":"Mouiche","year":"2025","journal-title":"Comput. Secur."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"103824","DOI":"10.1016\/j.cose.2024.103824","article-title":"KnowCTI: Knowledge-Based Cyber Threat Intelligence Entity and Relation Extraction","volume":"141","author":"Wang","year":"2024","journal-title":"Comput. Secur."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"118806","DOI":"10.1016\/j.eswa.2022.118806","article-title":"Learning Knowledge Graph Embedding with a Dual-Attention Embedding Network","volume":"212","author":"Fang","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"e341","DOI":"10.7717\/peerj-cs.341","article-title":"Application and Evaluation of Knowledge Graph Embeddings in Biomedical Data","volume":"7","author":"Alshahrani","year":"2021","journal-title":"PeerJ Comput. Sci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"101817","DOI":"10.1016\/j.artmed.2020.101817","article-title":"Real-World Data Medical Knowledge Graph: Construction and Applications","volume":"103","author":"Li","year":"2020","journal-title":"Artif. Intell. Med."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1016\/j.sbi.2021.09.003","article-title":"Toward Better Drug Discovery with Knowledge Graph","volume":"72","author":"Zeng","year":"2022","journal-title":"Curr. Opin. Struct. Biol."}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/12\/3\/100\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:47:03Z","timestamp":1760035623000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/12\/3\/100"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,22]]},"references-count":34,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["informatics12030100"],"URL":"https:\/\/doi.org\/10.3390\/informatics12030100","relation":{},"ISSN":["2227-9709"],"issn-type":[{"value":"2227-9709","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,22]]}}}