{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T18:18:35Z","timestamp":1776881915209,"version":"3.51.2"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2024,3,18]],"date-time":"2024-03-18T00:00:00Z","timestamp":1710720000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,18]],"date-time":"2024-03-18T00:00:00Z","timestamp":1710720000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001823","name":"Ministerstvo \u0160kolstv\u00ed, Ml\u00e1de\u017ee a T\u011blov\u00fdchovy","doi-asserted-by":"publisher","award":["SGS23\/211\/OHK3\/3T\/18"],"award-info":[{"award-number":["SGS23\/211\/OHK3\/3T\/18"]}],"id":[{"id":"10.13039\/501100001823","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001823","name":"Ministerstvo \u0160kolstv\u00ed, Ml\u00e1de\u017ee a T\u011blov\u00fdchovy","doi-asserted-by":"publisher","award":["SGS23\/211\/OHK3\/3T\/18"],"award-info":[{"award-number":["SGS23\/211\/OHK3\/3T\/18"]}],"id":[{"id":"10.13039\/501100001823","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100018240","name":"Research Center for Informatics, Czech Technical University in Prague","doi-asserted-by":"publisher","award":["CZ.02.1.01\/0.0\/0.0\/16 019\/0000765"],"award-info":[{"award-number":["CZ.02.1.01\/0.0\/0.0\/16 019\/0000765"]}],"id":[{"id":"10.13039\/100018240","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Comput Virol Hack Tech"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Because of its world-class results, machine learning (ML) is becoming increasingly popular as a go-to solution for many tasks. As a result, antivirus developers are incorporating ML models into their toolchains. While these models improve malware detection capabilities, they also carry the disadvantage of being susceptible to adversarial attacks. Although this vulnerability has been demonstrated for many models in white-box settings, a black-box scenario is more applicable in practice for the domain of malware detection. We present a method of creating adversarial malware examples using reinforcement learning algorithms. The reinforcement learning agents utilize a set of functionality-preserving modifications, thus creating valid adversarial examples. Using the proximal policy optimization (PPO) algorithm, we achieved an evasion rate of 53.84% against the gradient-boosted decision tree (GBDT) detector. The PPO agent previously trained against the GBDT classifier scored an evasion rate of 11.41% against the neural network-based classifier MalConv and an average evasion rate of 2.31% against top antivirus programs. Furthermore, we discovered that random application of our functionality-preserving portable executable modifications successfully evades leading antivirus engines, with an average evasion rate of 11.65%. These findings indicate that ML-based models used in malware detection systems are sensitive to adversarial attacks and that better safeguards need to be taken to protect these systems.\n<\/jats:p>","DOI":"10.1007\/s11416-024-00516-2","type":"journal-article","created":{"date-parts":[[2024,3,18]],"date-time":"2024-03-18T03:01:45Z","timestamp":1710730905000},"page":"607-621","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Creating valid adversarial examples of malware"],"prefix":"10.1007","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8329-7572","authenticated-orcid":false,"given":"Matou\u0161","family":"Koz\u00e1k","sequence":"first","affiliation":[]},{"given":"Martin","family":"Jure\u010dek","sequence":"additional","affiliation":[]},{"given":"Mark","family":"Stamp","sequence":"additional","affiliation":[]},{"given":"Fabio Di","family":"Troia","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,3,18]]},"reference":[{"key":"516_CR1","unstructured":"Institute, A.-T.: Malware statistics & trends report: AV-TEST (2022). https:\/\/www.av-test.org\/en\/statistics\/malware\/"},{"key":"516_CR2","unstructured":"Sophos: Sophos Threat Report (2022). https:\/\/www.sophos.com\/en-us\/content\/security-threat-report"},{"key":"516_CR3","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1016\/j.cose.2018.11.001","volume":"81","author":"D Ucci","year":"2019","unstructured":"Ucci, D., Aniello, L., Baldoni, R.: Survey of machine learning techniques for malware analysis. Comput. Secur. 81, 123\u2013147 (2019). https:\/\/doi.org\/10.1016\/j.cose.2018.11.001","journal-title":"Comput. Secur."},{"key":"516_CR4","doi-asserted-by":"publisher","unstructured":"Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z.B., Swami, A.: The limitations of deep learning in adversarial settings. In: 2016 IEEE European Symposium on Security and Privacy (EuroS &P), pp. 372\u2013387 (2016). https:\/\/doi.org\/10.1109\/EuroSP.2016.36. IEEE","DOI":"10.1109\/EuroSP.2016.36"},{"key":"516_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11416-015-0261-z","volume":"13","author":"A Damodaran","year":"2017","unstructured":"Damodaran, A., Troia, F.D., Visaggio, C.A., Austin, T.H., Stamp, M.: A comparison of static, dynamic, and hybrid analysis for malware detection. J. Comput. Virol. Hack. Tech. 13, 1\u201312 (2017). https:\/\/doi.org\/10.1007\/s11416-015-0261-z","journal-title":"J. Comput. Virol. Hack. Tech."},{"key":"516_CR6","unstructured":"Erko, A.: Malware sandbox evasion: techniques, principles and solutions (2022). https:\/\/www.apriorit.com\/dev-blog\/545-sandbox-evading-malware"},{"key":"516_CR7","unstructured":"Yuceel, H.C.: Virtualization\/sandbox evasion\u2014how attackers avoid malware analysis. Picus G\u00fcvenlik A.\u015e (2022). https:\/\/www.picussecurity.com\/resource\/virtualization\/sandbox-evasion-how-attackers-avoid-malware-analysis"},{"issue":"4","key":"516_CR8","first-page":"5","volume":"9","author":"A Kerckhoffs","year":"1883","unstructured":"Kerckhoffs, A.: La cryptographie militaire. J. Sci. Militaires 9(4), 5\u201338 (1883)","journal-title":"J. Sci. Militaires"},{"key":"516_CR9","doi-asserted-by":"publisher","unstructured":"Huang, L., Joseph, A.D., Nelson, B., Rubinstein, B.I.P., Tygar, J.D.: Adversarial machine learning. In: Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence. AISec \u201911, pp. 43\u201358. Association for Computing Machinery, New York, NY, USA (2011). https:\/\/doi.org\/10.1145\/2046684.2046692","DOI":"10.1145\/2046684.2046692"},{"key":"516_CR10","doi-asserted-by":"publisher","unstructured":"Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press (2018). https:\/\/doi.org\/10.1016\/S1364-6613(99)01331-5","DOI":"10.1016\/S1364-6613(99)01331-5"},{"key":"516_CR11","unstructured":"Watkins, C.J.C.H.: Learning from delayed rewards. King\u2019s College, Cambridge United Kingdom (1989). https:\/\/www.researchgate.net\/publication\/33784417_Learning_From_Delayed_Rewards"},{"key":"516_CR12","doi-asserted-by":"publisher","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning. CoRR arXiv:1312.5602 (2013). https:\/\/doi.org\/10.48550\/ARXIV.1312.5602","DOI":"10.48550\/ARXIV.1312.5602"},{"issue":"7540","key":"516_CR13","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529\u2013533 (2015). https:\/\/doi.org\/10.1038\/nature14236","journal-title":"Nature"},{"key":"516_CR14","unstructured":"Sutton, R.S., McAllester, D., Singh, S., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: Solla, S., Leen, T., M\u00fcller, K. (eds.) Proceedings of the 12th International Conference on Neural Information Processing Systems. NIPS\u201999, vol. 12, pp. 1057\u20131063. MIT Press, Cambridge, MA, USA (1999). https:\/\/proceedings.neurips.cc\/paper\/1999\/file\/464d828b85b0bed98e80ade0a5c43b0f-Paper.pdf"},{"key":"516_CR15","doi-asserted-by":"publisher","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. CoRR arXiv:1707.06347 (2017). https:\/\/doi.org\/10.48550\/arXiv.1707.06347","DOI":"10.48550\/arXiv.1707.06347"},{"key":"516_CR16","unstructured":"Kowalczyk, K.: Portable Executable File Format (2018). https:\/\/blog.kowalczyk.info\/articles\/pefileformat.html"},{"key":"516_CR17","unstructured":"Karl\u00a0Bridge, M.: PE Format - Win32 apps (2019). https:\/\/docs.microsoft.com\/en-us\/windows\/win32\/debug\/pe-format"},{"key":"516_CR18","unstructured":"Pietrek, M.: An In-Depth Look into the Win32 Portable Executable File Format (2008). https:\/\/docs.microsoft.com\/en-us\/previous-versions\/bb985992(v=msdn.10)?redirectedfrom=MSDN"},{"key":"516_CR19","doi-asserted-by":"publisher","unstructured":"Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: 3rd International Conference on Learning Representations (ICLR) (2015). https:\/\/doi.org\/10.48550\/ARXIV.1412.6572. arxiv:1412.6572","DOI":"10.48550\/ARXIV.1412.6572"},{"key":"516_CR20","doi-asserted-by":"publisher","unstructured":"Anderson, H.S., Kharkar, A., Filar, B., Evans, D., Roth, P.: Learning to evade static pe machine learning malware models via reinforcement learning. CoRR arXiv:1801.08917 (2018). https:\/\/doi.org\/10.48550\/arXiv.1801.08917","DOI":"10.48550\/arXiv.1801.08917"},{"issue":"4","key":"516_CR21","doi-asserted-by":"publisher","first-page":"0231626","DOI":"10.1371\/journal.pone.0231626","volume":"15","author":"Y Fang","year":"2020","unstructured":"Fang, Y., Zeng, Y., Li, B., Liu, L., Zhang, L.: Deepdetectnet vs rlattacknet: an adversarial method to improve deep learning-based static malware detection model. Plos one 15(4), 0231626 (2020). https:\/\/doi.org\/10.1371\/journal.pone.0231626","journal-title":"Plos one"},{"key":"516_CR22","doi-asserted-by":"publisher","unstructured":"Song, W., Li, X., Afroz, S., Garg, D., Kuznetsov, D., Yin, H.: Mab-malware: a reinforcement learning framework for attacking static malware classifiers. arXiv preprint arXiv:2003.03100 (2020). https:\/\/doi.org\/10.48550\/ARXIV.2003.03100","DOI":"10.48550\/ARXIV.2003.03100"},{"key":"516_CR23","doi-asserted-by":"publisher","unstructured":"Quertier, T., Marais, B., Morucci, S., Fournel, B.: Merlin\u2013malware evasion with reinforcement learning. arXiv preprint (2022). https:\/\/doi.org\/10.48550\/ARXIV.2203.12980arXiv:2203.12980","DOI":"10.48550\/ARXIV.2203.12980"},{"key":"516_CR24","doi-asserted-by":"publisher","unstructured":"Kolosnjaji, B., Demontis, A., Biggio, B., Maiorca, D., Giacinto, G., Eckert, C., Roli, F.: Adversarial malware binaries: Evading deep learning for malware detection in executables. In: 2018 26th European Signal Processing Conference (EUSIPCO), pp. 533\u2013537 (2018). https:\/\/doi.org\/10.23919\/EUSIPCO.2018.8553214 . IEEE. arXiv:1803.04173","DOI":"10.23919\/EUSIPCO.2018.8553214"},{"key":"516_CR25","doi-asserted-by":"publisher","unstructured":"Kreuk, F., Barak, A., Aviv-Reuven, S., Baruch, M., Pinkas, B., Keshet, J.: Deceiving end-to-end deep learning malware detectors using adversarial examples. CoRR arXiv:1802.04528 (2019) https:\/\/doi.org\/10.48550\/ARXIV.1802.04528","DOI":"10.48550\/ARXIV.1802.04528"},{"key":"516_CR26","doi-asserted-by":"publisher","unstructured":"Demetrio, L., Biggio, B., Lagorio, G., Roli, F., Armando, A.: Explaining vulnerabilities of deep learning to adversarial malware binaries. arXiv:1901.03583 (2019) https:\/\/doi.org\/10.48550\/ARXIV.1901.03583","DOI":"10.48550\/ARXIV.1901.03583"},{"issue":"1","key":"516_CR27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s42400-021-00079-5","volume":"4","author":"C Yang","year":"2021","unstructured":"Yang, C., Xu, J., Liang, S., Wu, Y., Wen, Y., Zhang, B., Meng, D.: Deepmal: maliciousness-preserving adversarial instruction learning against static malware detection. Cybersecurity 4(1), 1\u201314 (2021). https:\/\/doi.org\/10.1186\/s42400-021-00079-5","journal-title":"Cybersecurity"},{"key":"516_CR28","doi-asserted-by":"publisher","unstructured":"Hu, W., Tan, Y.: Generating adversarial malware examples for black-box attacks based on gan. CoRR arXiv:1702.05983 (2017). https:\/\/doi.org\/10.48550\/ARXIV.1702.05983","DOI":"10.48550\/ARXIV.1702.05983"},{"key":"516_CR29","doi-asserted-by":"publisher","unstructured":"Ebrahimi, M., Zhang, N., Hu, J., Raza, M.T., Chen, H.: Binary black-box evasion attacks against deep learning-based static malware detectors with adversarial byte-level language model. CoRR arXiv:2012.07994 (2020). https:\/\/doi.org\/10.48550\/ARXIV.2012.07994","DOI":"10.48550\/ARXIV.2012.07994"},{"key":"516_CR30","doi-asserted-by":"publisher","first-page":"3469","DOI":"10.1109\/TIFS.2021.3082330","volume":"16","author":"L Demetrio","year":"2021","unstructured":"Demetrio, L., Biggio, B., Lagorio, G., Roli, F., Armando, A.: Functionality-preserving black-box optimization of adversarial windows malware. IEEE Trans. Inf. Forensics Secur. 16, 3469\u20133478 (2021). https:\/\/doi.org\/10.1109\/TIFS.2021.3082330","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"516_CR31","doi-asserted-by":"publisher","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym. CoRR arXiv:1606.01540. https:\/\/doi.org\/10.48550\/ARXIV.1606.01540 (2016)","DOI":"10.48550\/ARXIV.1606.01540"},{"key":"516_CR32","doi-asserted-by":"publisher","unstructured":"Anderson, H.S., Roth, P.: Ember: an open dataset for training static pe malware machine learning models. CoRR arXiv:1804.04637 (2018). https:\/\/doi.org\/10.48550\/ARXIV.1804.04637","DOI":"10.48550\/ARXIV.1804.04637"},{"key":"516_CR33","doi-asserted-by":"publisher","unstructured":"Raff, E., Barker, J., Sylvester, J., Brandon, R., Catanzaro, B., Nicholas, C.: Malware detection by eating a whole exe (2017). https:\/\/doi.org\/10.48550\/ARXIV.1710.09435","DOI":"10.48550\/ARXIV.1710.09435"},{"key":"516_CR34","doi-asserted-by":"publisher","first-page":"326","DOI":"10.1016\/j.cose.2017.11.007","volume":"73","author":"S Chen","year":"2018","unstructured":"Chen, S., Xue, M., Fan, L., Hao, S., Xu, L., Zhu, H., Li, B.: Automated poisoning attacks and defenses in malware detection systems: an adversarial machine learning approach. Comput. Secur. 73, 326\u2013344 (2018). https:\/\/doi.org\/10.1016\/j.cose.2017.11.007","journal-title":"Comput. Secur."},{"key":"516_CR35","unstructured":"Thomas, R.: LIEF\u2014Library to Instrument Executable Formats (2017). https:\/\/lief.quarkslab.com\/"},{"key":"516_CR36","unstructured":"Carrera, E.: Pefile (2017). https:\/\/github.com\/erocarrera\/pefile"},{"key":"516_CR37","unstructured":"Guarnieri, C.: Cuckoo Sandbox\u2014Automated Malware Analysis (2012). https:\/\/cuckoosandbox.org\/"},{"key":"516_CR38","doi-asserted-by":"publisher","unstructured":"Liang, E., Liaw, R., Nishihara, R., Moritz, P., Fox, R., Gonzalez, J., Goldberg, K., Stoica, I.: Ray rllib: A composable and scalable reinforcement learning library. CoRR arXiv:1712.09381 (2017). https:\/\/doi.org\/10.48550\/arXiv.1712.09381","DOI":"10.48550\/arXiv.1712.09381"},{"key":"516_CR39","unstructured":"rukaimi: PE Bliss, Cross-Platform Portable Executable C++ Library. GitHub (2012). https:\/\/github.com\/BackupGGCode\/portable-executable-library"},{"key":"516_CR40","unstructured":"IBM: what is overfitting? (2022). https:\/\/www.ibm.com\/topics\/overfitting"},{"key":"516_CR41","unstructured":"AV-Comparatives: Malware Protection Test March 2023 (2023). https:\/\/www.av-comparatives.org\/tests\/malware-protection-test-march-2023\/"}],"container-title":["Journal of Computer Virology and Hacking Techniques"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11416-024-00516-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11416-024-00516-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11416-024-00516-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,15]],"date-time":"2024-10-15T15:07:50Z","timestamp":1729004870000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11416-024-00516-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,18]]},"references-count":41,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["516"],"URL":"https:\/\/doi.org\/10.1007\/s11416-024-00516-2","relation":{},"ISSN":["2263-8733"],"issn-type":[{"value":"2263-8733","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,18]]},"assertion":[{"value":"9 September 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 February 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 March 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no relevant financial or non-financial interests to disclose.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}