{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T11:32:28Z","timestamp":1782300748492,"version":"3.54.5"},"reference-count":51,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,3,26]],"date-time":"2021-03-26T00:00:00Z","timestamp":1616716800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/K032623\/1"],"award-info":[{"award-number":["EP\/K032623\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Malware detection is in a coevolutionary arms race where the attackers and defenders are constantly seeking advantage. This arms race is asymmetric: detection is harder and more expensive than evasion. White hats must be conservative to avoid false positives when searching for malicious behaviour. We seek to redress this imbalance. Most of the time, black hats need only make incremental changes to evade them. On occasion, white hats make a disruptive move and find a new technique that forces black hats to work harder. Examples include system calls, signatures and machine learning. We present a method, called Hothouse, that combines simulation and search to accelerate the white hat\u2019s ability to counter the black hat\u2019s incremental moves, thereby forcing black hats to perform disruptive moves more often. To realise Hothouse, we evolve EEE, an entropy-based polymorphic packer for Windows executables. Playing the role of a black hat, EEE uses evolutionary computation to disrupt the creation of malware signatures. We enter EEE into the detection arms race with VirusTotal, the most prominent cloud service for running anti-virus tools on software. During our 6 month study, we continually improved EEE in response to VirusTotal, eventually learning a packer that produces packed malware whose evasiveness goes from an initial 51.8% median to 19.6%. We report both how well VirusTotal learns to detect EEE-packed binaries and how well VirusTotal forgets in order to reduce false positives. VirusTotal\u2019s tools learn and forget fast, actually in about 3 days. We also show where VirusTotal focuses its detection efforts, by analysing EEE\u2019s variants.<\/jats:p>","DOI":"10.3390\/e23040395","type":"journal-article","created":{"date-parts":[[2021,3,26]],"date-time":"2021-03-26T13:17:53Z","timestamp":1616764673000},"page":"395","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Getting Ahead of the Arms Race: Hothousing the Coevolution of VirusTotal with a Packer"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6314-3725","authenticated-orcid":false,"given":"H\u00e9ctor D.","family":"Men\u00e9ndez","sequence":"first","affiliation":[{"name":"Computer Science Department, Middlesex University London, London NW4 4BG, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Clark","sequence":"additional","affiliation":[{"name":"Computer Science Department, University College London, London WC1E 6BT, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Earl","family":"T. Barr","sequence":"additional","affiliation":[{"name":"Computer Science Department, University College London, London WC1E 6BT, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1016\/j.eswa.2017.11.032","article-title":"Picking on the family: Disrupting android malware triage by forcing misclassification","volume":"95","author":"Calleja","year":"2018","journal-title":"Expert Syst. Appl."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1016\/j.eswa.2018.10.011","article-title":"The arms race: Adversarial search defeats entropy used to detect malware","volume":"118","author":"Bhattacharya","year":"2019","journal-title":"Expert Syst. Appl."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1145\/1387673.1387674","article-title":"A semantics-based approach to malware detection","volume":"30","author":"Preda","year":"2008","journal-title":"ACM Trans. Program. Lang. Syst."},{"key":"ref_4","unstructured":"Xu, W., Qi, Y., and Evans, D. (2016, January 25\u201327). Automatically evading classifiers. Proceedings of the 2016 Network and Distributed Systems Symposium, New Delhi, India."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/MSP.2004.100","article-title":"How to win an evolutionary arms race","volume":"2","author":"Somayaji","year":"2004","journal-title":"IEEE Secur. Priv."},{"key":"ref_6","unstructured":"Man, K.F., Tang, K.S., and Kwong, S. (2012). Genetic Algorithms: Concepts and Designs, Springer Science & Business Media."},{"key":"ref_7","unstructured":"Total, V. (2016, January 31). VirusTotal-Free Online Virus, Malware and URL Scanner. Available online: https:\/\/www.virustotal.com."},{"key":"ref_8","first-page":"4","article-title":"Protecting software through obfuscation: Can it keep pace with progress in code analysis?","volume":"49","author":"Schrittwieser","year":"2016","journal-title":"ACM Comput. Surv."},{"key":"ref_9","unstructured":"Guerra, P.H.C., Guedes, D., Meira, J.W., Hoepers, C., Chaves, M., and Steding-Jessen, K. (2010, January 13\u201314). Exploring the spam arms race to characterize spam evolution. Proceedings of the 7th Collaboration, Electronic messaging, Anti-Abuse and Spam Conference (CEAS), Redmond, WA, USA."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Marsden, C.T. (2010). Net Neutrality: Towards a Co-Regulatory Solution, A&C Black.","DOI":"10.5040\/9781849662192"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"e1259","DOI":"10.1002\/widm.1259","article-title":"A survey of game theoretic approach for adversarial machine learning","volume":"9","author":"Zhou","year":"2019","journal-title":"Wiley Interdiscip. Rev. Data Min. Knowl. Discov."},{"key":"ref_12","unstructured":"Gibbons, R. (1992). A primer in Game Theory, Harvester Wheatsheaf."},{"key":"ref_13","first-page":"59:1","article-title":"On the Security of Machine Learning in Malware C& C Detection: A Survey","volume":"49","author":"Gardiner","year":"2016","journal-title":"ACM Comput. Surv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Srndic, N., and Laskov, P. (2014, January 18\u201321). Practical Evasion of a Learning-Based Classifier: A Case Study. Proceedings of the 2014 IEEE Symposium on Security and Privacy, Berkeley, CA, USA.","DOI":"10.1109\/SP.2014.20"},{"key":"ref_15","unstructured":"Malin, C.H., Casey, E., and Aquilina, J.M. (2013). Malware Forensics Field Guide for Linux Systems: Digital Forensics Field Guides, Newnes."},{"key":"ref_16","first-page":"1601","article-title":"Moa: Massive online analysis","volume":"11","author":"Bifet","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1080\/00031305.1998.10480559","article-title":"Violin plots: A box plot-density trace synergism","volume":"52","author":"Hintze","year":"1998","journal-title":"Am. Stat."},{"key":"ref_18","first-page":"2008","article-title":"The art of unpacking","volume":"12","author":"Yason","year":"2007","journal-title":"Retrieved Feb"},{"key":"ref_19","unstructured":"Oberhumer, M., Moln\u00e1r, L., and Reiser, J.F. (2015, June 01). UPX: The Ultimate Packer for eXecutables. Available online: http:\/\/upx.sourceforge.net\/."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Globerson, A., and Roweis, S. (2006, January 18\u201321). Nightmare at test time: Robust learning by feature deletion. Proceedings of the 23rd International Conference on Machine Learning, San Jose, CA, USA.","DOI":"10.1145\/1143844.1143889"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Kantarcioglu, M., Thuraisingham, B., and Xi, B. (2012, January 12\u201316). Adversarial support vector machine learning. Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Beijing, China.","DOI":"10.1145\/2339530.2339697"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1007\/s10618-010-0197-3","article-title":"Classifier evaluation and attribute selection against active adversaries","volume":"22","author":"Xi","year":"2011","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1007\/s10994-010-5199-2","article-title":"Mining adversarial patterns via regularized loss minimization","volume":"81","author":"Liu","year":"2010","journal-title":"Mach. Learn."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Dua, S., and Du, X. (2016). Data Mining and Machine Learning in Cybersecurity, Auerbach Publications.","DOI":"10.1201\/b10867"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1016\/j.patcog.2011.06.019","article-title":"A unifying view on dataset shift in classification","volume":"45","author":"Raeder","year":"2012","journal-title":"Pattern Recognit."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"7405","DOI":"10.1007\/s00500-016-2283-y","article-title":"MOCDroid: Multi-objective evolutionary classifier for Android malware detection","volume":"21","author":"Camacho","year":"2017","journal-title":"Soft Comput."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Feng, Y., Anand, S., Dillig, I., and Aiken, A. (2014, January 16\u201322). Apposcopy: Semantics-based Detection of Android Malware Through Static Analysis. Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, Hong Kong, China.","DOI":"10.1145\/2635868.2635869"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Sebasti\u00e1n, M., Rivera, R., Kotzias, P., and Caballero, J. (2016, January 10\u201312). Avclass: A tool for massive malware labeling. Proceedings of the International Symposium on Research in Attacks, Intrusions, and Defenses, Heraklion, Greece.","DOI":"10.1007\/978-3-319-45719-2_11"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Men\u00e9ndez, H.D., and Llorente, J.L. (2019). Mimicking Anti-Viruses with Machine Learning and Entropy Profiles. Entropy, 21.","DOI":"10.3390\/e21050513"},{"key":"ref_30","unstructured":"Hammad, M., Garcia, J., and Malek, S. (June, January 27). A large-scale empirical study on the effects of code obfuscations on Android apps and anti-malware products. Proceedings of the 40th International Conference on Software Engineering, Gothenburg, Sweden."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Chinavle, D., Kolari, P., Oates, T., and Finin, T. (2009, January 2\u20136). Ensembles in adversarial classification for spam. Proceedings of the 18th ACM Conference on Information and Knowledge Management, Hong Kong, China.","DOI":"10.1145\/1645953.1646290"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Smutz, C., and Stavrou, A. (2012, January 3\u20137). Malicious PDF detection using metadata and structural features. Proceedings of the 28th Annual Computer Security Applications Conference, Orlando, FL, USA.","DOI":"10.1145\/2420950.2420987"},{"key":"ref_33","unstructured":"\u0160rndic, N., and Laskov, P. (2013, January 24\u201327). Detection of malicious pdf files based on hierarchical document structure. Proceedings of the 20th Annual Network & Distributed System Security Symposium, San Diego, CA, USA."},{"key":"ref_34","unstructured":"Biggio, B., Nelson, B., and Laskov, P. (July, January 26). Poisoning Attacks against Support Vector Machines. Proceedings of the 29th International Conference on Machine Learning, Edinburgh, UK."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"984","DOI":"10.1109\/TKDE.2013.57","article-title":"Security evaluation of pattern classifiers under attack","volume":"26","author":"Biggio","year":"2014","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"M\u00e9nendez, H.D. (2019, January 10\u201313). VARMOG: A Co-Evolutionary Algorithm to Identify Manifolds on Large Data. Proceedings of the 2019 IEEE Congress on Evolutionary Computation (CEC), Wellington, New Zealand.","DOI":"10.1109\/CEC.2019.8790004"},{"key":"ref_37","first-page":"814","article-title":"Designing large quantum key distribution networks via medoid-based algorithms","volume":"115","year":"2020","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/584091.584093","article-title":"A mathematical theory of communication","volume":"5","author":"Shannon","year":"2001","journal-title":"ACM Sigmobile Mob. Comput. Commun. Rev."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1109\/MSP.2007.48","article-title":"Using entropy analysis to find encrypted and packed malware","volume":"5","author":"Lyda","year":"2007","journal-title":"IEEE Secur. Priv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Shafiq, M.Z., Tabish, S., and Farooq, M. (2009, January 23\u201325). PE-probe: Leveraging packer detection and structural information to detect malicious portable executables. Proceedings of the Virus Bulletin Conference (VB), Citeseer, Switzerland.","DOI":"10.1007\/978-3-642-04342-0_7"},{"key":"ref_41","unstructured":"McMillan, C., and Garman, J. (2011). System and Method for Determining Data Entropy to Identify Malware. (8,069,484), US Patent."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Alshahwan, N., T Barr, E., Clark, D., Danezis, G., and D Men\u00e9ndez, H. (2020). Detecting malware with information complexity. Entropy, 22.","DOI":"10.3390\/e22050575"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Zhou, Y., and Inge, W.M. (2008, January 27\u201331). Malware Detection Using Adaptive Data Compression. Proceedings of the 1st ACM Workshop on Workshop on AISec, Alexandria, VA, USA.","DOI":"10.1145\/1456377.1456393"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Mart\u00edn, A., Calleja, A., Men\u00e9ndez, H.D., Tapiador, J., and Camacho, D. (2016, January 6\u20139). ADROIT: Android malware detection using meta-information. Proceedings of the 2016 IEEE Symposium Series on Computational Intelligence (SSCI), Athens, Greece.","DOI":"10.1109\/SSCI.2016.7849904"},{"key":"ref_45","unstructured":"Shafiq, M.Z., Khayam, S.A., and Farooq, M. (2008, January 10\u201311). Embedded malware detection using markov n-grams. Proceedings of the International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, Lisbon, Portugal."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Perdisci, R., Lanzi, A., and Lee, W. (2008, January 8\u201312). McBoost: Boosting scalability in malware collection and analysis using statistical classification of executables. Proceedings of the Annual Computer Security Applications Conference, Anaheim, CA, USA.","DOI":"10.1109\/ACSAC.2008.22"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Tabish, S.M., Shafiq, M.Z., and Farooq, M. (2009, January 28). Malware Detection Using Statistical Analysis of Byte-level File Content. Proceedings of the ACM SIGKDD Workshop on CyberSecurity and Intelligence Informatics, Paris, France.","DOI":"10.1145\/1599272.1599278"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Ugarte-Pedrero, X., Santos, I., Sanz, B., Laorden, C., and Bringas, P.G. (2012, January 14\u201317). Countering entropy measure attacks on packed software detection. Proceedings of the Consumer Communications and Networking Conference (CCNC), Las Vegas, NV, USA.","DOI":"10.1109\/CCNC.2012.6181079"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Jacob, G., Comparetti, P.M., Neugschwandtner, M., Kruegel, C., and Vigna, G. (2012, January 26\u201327). A static, packer-agnostic filter to detect similar malware samples. Proceedings of the International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, Heraklion, Greece.","DOI":"10.1007\/978-3-642-37300-8_6"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1007\/s11416-011-0153-9","article-title":"Comparing files using structural entropy","volume":"7","author":"Sorokin","year":"2011","journal-title":"J. Comput. Virol."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1007\/s11416-013-0185-4","article-title":"Structural entropy and metamorphic malware","volume":"9","author":"Baysa","year":"2013","journal-title":"J. Comput. Virol. Hacking Tech."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/4\/395\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:32:50Z","timestamp":1760362370000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/4\/395"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,26]]},"references-count":51,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,4]]}},"alternative-id":["e23040395"],"URL":"https:\/\/doi.org\/10.3390\/e23040395","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,26]]}}}