{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T14:48:55Z","timestamp":1784645335493,"version":"3.55.0"},"reference-count":57,"publisher":"Association for Computing Machinery (ACM)","issue":"3","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Digital Threats"],"published-print":{"date-parts":[[2025,9,30]]},"abstract":"<jats:p>This article investigates the application of Deep Reinforcement Learning (DRL) for attributing malware to specific Advanced Persistent Threat (APT) groups through detailed behavioural analysis. By analysing over 3,500 malware samples from 12 distinct APT groups, the study utilises sophisticated tools like Cuckoo Sandbox to extract behavioural data, providing a deep insight into the operational patterns of malware. The research demonstrates that the DRL model significantly outperforms traditional machine learning approaches such as SGD, SVC, KNN, MLP and Decision Tree Classifiers, achieving an impressive test accuracy of 94.12%. It highlights the model\u2019s capability to adeptly manage complex, variable and elusive malware attributes. Furthermore, the article discusses the considerable computational resources and extensive data dependencies required for deploying these advanced AI models in cybersecurity frameworks. Future research is directed towards enhancing the efficiency of DRL models, expanding the diversity of the datasets, addressing ethical concerns and leveraging Large Language Models (LLMs) to refine reward mechanisms and optimise the DRL framework. By showcasing the transformative potential of DRL in malware attribution, this research advocates for a responsible and balanced approach to AI integration, with the goal of advancing cybersecurity through more adaptable, accurate and robust systems.<\/jats:p>","DOI":"10.1145\/3736654","type":"journal-article","created":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T13:02:16Z","timestamp":1747746136000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Advanced Persistent Threats (APT) Attribution Using Deep Reinforcement Learning"],"prefix":"10.1145","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-4588-1345","authenticated-orcid":false,"given":"Animesh Singh","family":"Basnet","sequence":"first","affiliation":[{"name":"Cyber Security Research Centre, London Metropolitan University, London, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7067-7848","authenticated-orcid":false,"given":"Mohamed Chahine","family":"Ghanem","sequence":"additional","affiliation":[{"name":"Cybersecurity Institute, Department of Computer Science, University of Liverpool, Liverpool, UK, and Cyber Security Research Centre, London Metropolitan University, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-7376-0477","authenticated-orcid":false,"given":"Dipo","family":"Dunsin","sequence":"additional","affiliation":[{"name":"Cyber Security Research Centre, London Metropolitan University, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9532-2453","authenticated-orcid":false,"given":"Hamza","family":"Kheddar","sequence":"additional","affiliation":[{"name":"LSEA Laboratory, University of Medea, Medea, Algeria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2383-6815","authenticated-orcid":false,"given":"Wiktor","family":"Sowinski-Mydlarz","sequence":"additional","affiliation":[{"name":"Cyber Security Research Centre, London Metropolitan University, London, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,9,29]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"Stable Baselines 3. 2024. Stable-Baselines3 Docs\u2014Reliable Reinforcement Learning Implementations & x2014; Stable Baselines3 2.4.0a9 Documentation\u2014stable-baselines3.readthedocs.io. Retrieved August 20 2024 from https:\/\/stable-baselines3.readthedocs.io\/en\/master\/"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2017.2743240"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.55529\/ijitc.43.23.35"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.3929\/ethz-b-000200661"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3696014"},{"key":"e_1_3_1_7_2","unstructured":"cuckoosandbox. 2024. Cuckoo Sandbox\u2014Automated Malware Analysis\u2014cuckoosandbox.org. Retrieved October 11 2024 from https:\/\/www.cuckoo.ee\/"},{"key":"e_1_3_1_8_2","unstructured":"CyberResearch. 2019. GitHub\u2014Cyber-Research\/APTMalware: APT Malware Dataset Containing over 3 500 State-Sponsored Malware Samples\u2014github.com. Retrieved October 5 2024 from https:\/\/github.com\/cyber-research\/APTMalware"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0305618"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.hcc.2025.100299"},{"key":"e_1_3_1_11_2","unstructured":"Mohammed Ashfaaq M. Farzaan Mohamed Chahine Ghanem Ayman El-Hajjar and Deepthi N. Ratnayake. 2024. AI-enabled system for efficient and effective cyber incident detection and response in cloud environments. arXiv:2404.05602. Retrieved from https:\/\/arxiv.org\/abs\/2404.05602"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.iswa.2025.200495"},{"key":"e_1_3_1_13_2","unstructured":"Farama Foundation. 2023. Gymnasium Documentation\u2014gymnasium.farama.org. Retrieved August 19 2024 from https:\/\/gymnasium.farama.org\/index.html"},{"key":"e_1_3_1_14_2","unstructured":"Neil Fox. 2023. Cuckoo Sandbox Overview\u2014varonis.com. Retrieved August 17 2024 from https:\/\/www.varonis.com\/blog\/cuckoo-sandbox"},{"key":"e_1_3_1_15_2","unstructured":"Mohamed Chahine Ghanem Elhadj Benkhelifa Dominik Wojtczak Mohamed Amine Ferrag Norbert Tihanyi and Erivelton Geraldo Nepomuceno. 2025. Leveraging reinforcement learning for an efficient automation of windows registry analysis during cyber incident response. TechRxiv. Retrieved January 07 2025 from 10.36227\/techrxiv.173626850.08639272\/v1"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3332834"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.3390\/jcp3040036"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.3390\/app132111908"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jnca.2018.10.022"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/s42979-023-01744-x"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","unstructured":"Mohammed Hassanin and Nour Moustafa. 2024. A comprehensive overview of large language models (LLMs) for cyber defences: Opportunities and directions. arXiv:2405.14487. DOI: 10.48550\/ARXIV.2405.14487","DOI":"10.48550\/ARXIV.2405.14487"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.62441\/nano-ntp.v20iS4.33"},{"key":"e_1_3_1_23_2","unstructured":"kasperskyLab. 2020. The Power of Threat Attribution: Challenges and Benefits of Cyberthreat Attribution\u2014kaspersky.com. Retrieved May 12 2024 from https:\/\/media.kaspersky.com\/en\/business-security\/enterprise\/threat-attribution-engine-whitepaper.pdf"},{"key":"e_1_3_1_24_2","unstructured":"Edward Kost. 2023. What Is an Advanced Persistent Threat (APT)?\u2014UpGuard\u2014upguard.com. Retrieved August 4 2024 from https:\/\/www.upguard.com\/blog\/what-is-an-advanced-persistent-threat"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2022.03.003"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1155\/2021\/9396141"},{"key":"e_1_3_1_27_2","unstructured":"Xiaoguang Li. 2023. Create Custom OpenAI Gym Environment for Deep Reinforcement Learning (drl4t-04). Retrieved August 15 2024 from https:\/\/lixiaoguang.medium.com\/create-custom-openai-gym-environment-for-deep-reinforcement-learning-drl-af2b2e3c830d"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2019.112963"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.3390\/computers14020067"},{"issue":"6","key":"e_1_3_1_30_2","doi-asserted-by":"crossref","first-page":"118","DOI":"10.3390\/computers12060118","article-title":"Unbalanced web phishing classification through deep reinforcement learning","volume":"12","author":"Maci Antonio","year":"2023","unstructured":"Antonio Maci, Alessandro Santorsola, Antonio Coscia, and Andrea Iannacone. 2023. Unbalanced web phishing classification through deep reinforcement learning. Computers 12, 6 (2023), 118. Retrieved from https:\/\/www.mdpi.com\/2073-431X\/12\/6\/118","journal-title":"Computers"},{"key":"e_1_3_1_31_2","unstructured":"Joaquin Matamis. 2024. Advancing Accountability in Cyberspace \u2022 Stimson Center\u2014stimson.org. Retrieved September 5 2024 from https:\/\/www.stimson.org\/2024\/advancing-accountability-in-cyberspace\/"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2024.3360260"},{"key":"e_1_3_1_33_2","unstructured":"Eduardo C. Garrido Merch\u00e1n. 2023. Why Deep Reinforcement Learning Is Going to Be the Next Big Deal in AI\u2014eduardogarrido90. Retrieved August 10 2024 from https:\/\/medium.com\/@eduardogarrido90\/why-deep-reinforcement-learning-is-going-to-be-the-next-big-deal-in-ai-2e796bdf47d2"},{"key":"e_1_3_1_34_2","unstructured":"[33] The Hacker News. 2024. 3 Ransomware Group Newcomers to Watch in 2024\u2014thehackernews.com. Retrieved August 6 2024 from https:\/\/thehackernews.com\/2024\/01\/3-ransomware-group-newcomers-to-watch.html"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics13030555"},{"key":"e_1_3_1_36_2","unstructured":"Juan C. Olamendy. 2023. Understanding ReLU LeakyReLU and PReLU: A Comprehensive Guide\u2014juanc.olamendy. Retrieved August 18 2024 from https:\/\/medium.com\/@juanc.olamendy\/understanding-relu-leakyrelu-and-prelu-a-comprehensive-guide-20f2775d3d64"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3011502"},{"key":"e_1_3_1_38_2","doi-asserted-by":"crossref","unstructured":"Nanda Rani Bikash Saha and Sandeep Kumar Shukla. 2024. A comprehensive survey of advanced persistent threat attribution: Taxonomy methods challenges and open research problems. arXiv:2409.11415. Retrieved from https:\/\/arxiv.org\/abs\/2409.11415","DOI":"10.70777\/agi.v1i1.10869"},{"key":"e_1_3_1_39_2","unstructured":"Muhammad Raza. 2023. What Are TTPs? Tactics Techniques & Procedures Explained\u2014Splunk\u2014splunk.com. Retrieved August 10 2024 from https:\/\/www.splunk.com\/en_us\/blog\/learn\/ttp-tactics-techniques-procedures.html"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-19366-3"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.3390\/e20050390"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/AICS60730.2023.10470498"},{"key":"e_1_3_1_43_2","unstructured":"Samet. 2022. What Is Virus Total?\u2014sametyorulmaz777. Retrieved October 11 2024 from https:\/\/medium.com\/@sametyorulmaz777\/what-is-virus-total-70c64b7c5e95"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jiixd.2023.12.001"},{"key":"e_1_3_1_45_2","unstructured":"scikitLearn. 2024. MinMaxScaler\u2014scikit-learn.org. Retrieved August 16 2024 from https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.preprocessing.MinMaxScaler.html"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10796-022-10333-x"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2024.103862"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN48605.2020.9207120"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/PRAI59366.2023.10331977"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compeleceng.2022.108261"},{"key":"e_1_3_1_51_2","unstructured":"Manasa Vemuri. 2020. Malware Analysis Using Cuckoo Sandbox\u2014easypeazyo14. Retrieved October 11 2024 from https:\/\/medium.com\/@easypeazyo14\/malware-analysis-using-cuckoo-sandbox-756616e6e85e"},{"key":"e_1_3_1_52_2","unstructured":"VirusTotal. 2024. VirusTotal\u2014virustotal.com. Retrieved October 11 2024 from https:\/\/www.virustotal.com\/gui\/intelligence-overview"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/MNET.2024.3389734"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1155\/2021\/8077220"},{"key":"e_1_3_1_55_2","doi-asserted-by":"crossref","unstructured":"Nan Xiao Bo Lang Ting Wang and Yikai Chen. 2024. APT-MMF: An advanced persistent threat actor attribution method based on multimodal and multilevel feature fusion. arXiv:2402.12743. Retrieved from https:\/\/arxiv.org\/abs\/2402.12743","DOI":"10.1016\/j.cose.2024.103960"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2014.2320725"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0304066"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/NETAPPS.2010.46"}],"container-title":["Digital Threats: Research and Practice"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3736654","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,29]],"date-time":"2025-09-29T16:10:37Z","timestamp":1759162237000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3736654"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,29]]},"references-count":57,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,9,30]]}},"alternative-id":["10.1145\/3736654"],"URL":"https:\/\/doi.org\/10.1145\/3736654","relation":{},"ISSN":["2692-1626","2576-5337"],"issn-type":[{"value":"2692-1626","type":"print"},{"value":"2576-5337","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,29]]},"assertion":[{"value":"2024-10-17","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-13","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-09-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}