{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T18:19:04Z","timestamp":1787249944087,"version":"3.56.0"},"reference-count":33,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2023,3,10]],"date-time":"2023-03-10T00:00:00Z","timestamp":1678406400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Research Program funded by SeoulTech (Seoul National University of Science and Technology)"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Cybersecurity is a growing concern in today\u2019s interconnected world. Traditional cybersecurity approaches, such as signature-based detection and rule-based firewalls, are often limited in their ability to effectively respond to evolving and sophisticated cyber threats. Reinforcement learning (RL) has shown great potential in solving complex decision-making problems in various domains, including cybersecurity. However, there are significant challenges to overcome, such as the lack of sufficient training data and the difficulty of modeling complex and dynamic attack scenarios hindering researchers\u2019 ability to address real-world challenges and advance the state of the art in RL cyber applications. In this work, we applied a deep RL (DRL) framework in adversarial cyber-attack simulation to enhance cybersecurity. Our framework uses an agent-based model to continuously learn from and adapt to the dynamic and uncertain environment of network security. The agent decides on the optimal attack actions to take based on the state of the network and the rewards it receives for its decisions. Our experiments on synthetic network security show that the DRL approach outperforms existing methods in terms of learning optimal attack actions. Our framework represents a promising step towards the development of more effective and dynamic cybersecurity solutions.<\/jats:p>","DOI":"10.3390\/s23063000","type":"journal-article","created":{"date-parts":[[2023,3,10]],"date-time":"2023-03-10T02:30:33Z","timestamp":1678415433000},"page":"3000","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["Applying Reinforcement Learning for Enhanced Cybersecurity against Adversarial Simulation"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1101-9041","authenticated-orcid":false,"given":"Sang Ho","family":"Oh","sequence":"first","affiliation":[{"name":"Business Department of Convergence and Open Sharing System, Seoul National University of Science and Technology, Seoul 01811, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Min Ki","family":"Jeong","sequence":"additional","affiliation":[{"name":"Department of Applied Artificial Intelligence, Seoul National University of Science and Technology, Seoul 01811, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hyung Chan","family":"Kim","sequence":"additional","affiliation":[{"name":"The Affiliated Institute of Electronics and Telecommunications Research Institute, Daejeon 34044, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4878-4129","authenticated-orcid":false,"given":"Jongyoul","family":"Park","sequence":"additional","affiliation":[{"name":"Business Department of Convergence and Open Sharing System, Seoul National University of Science and Technology, Seoul 01811, Republic of Korea"},{"name":"Department of Applied Artificial Intelligence, Seoul National University of Science and Technology, Seoul 01811, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,10]]},"reference":[{"key":"ref_1","unstructured":"Li, L., Fayad, R., and Taylor, A. (2021). Cygil: A Cyber Gym for Training Autonomous Agents over Emulated Network Systems. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Yoo, J.D., Park, E., Lee, G., Ahn, M.K., Kim, D., Seo, S., and Kim, H.K. (2020). Cyber Attack and Defense Emulation Agents. Appl. Sci., 10.","DOI":"10.3390\/app10062140"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1851","DOI":"10.1109\/COMST.2019.2891891","article-title":"A Survey on Advanced Persistent Threats: Techniques, Solutions, Challenges, and Research Opportunities","volume":"21","author":"Alshamrani","year":"2019","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_4","unstructured":"Haider, N., Baig, M.Z., and Imran, M. (2020). Artificial Intelligence and Machine Learning in 5G Network Security: Opportu-nities, advantages, and future research trends. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"527","DOI":"10.3390\/jcp2030027","article-title":"Cybersecurity Threats and Their Mitigation Approaches Using Machine Learning\u2014A Review","volume":"2","author":"Ahsan","year":"2022","journal-title":"J. Cybersecur. Priv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-020-00318-5","article-title":"Cybersecurity data science: An overview from machine learning perspective","volume":"7","author":"Sarker","year":"2020","journal-title":"J. Big Data"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Piplai, A., Anoruo, M., Fasaye, K., Joshi, A., Finin, T., and Ridley, A. (2022, January 12). Knowledge guided Two-player Rein-forcement Learning for Cyber Attacks and Defenses. Proceedings of the International Conference on Machine Learning and Applications, Miami, FL, USA.","DOI":"10.1109\/ICMLA55696.2022.00213"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"356","DOI":"10.14429\/dsj.68.12371","article-title":"A Survey of Adversarial Machine Learning in Cyber Warfare","volume":"68","author":"Duddu","year":"2018","journal-title":"Def. Sci. J."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Shaukat, K., Luo, S., Varadharajan, V., Hameed, I.A., Chen, S., Liu, D., and Li, J. (2020). Performance Comparison and Current Challenges of Using Machine Learning Techniques in Cybersecurity. Energies, 13.","DOI":"10.3390\/en13102509"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Applebaum, A., Miller, D., Strom, B., Korban, C., and Wolf, R. (2016, January 5\u20138). Intelligent, Automated Red Team Emulation. Proceedings of the 32nd Annual Conference on Computer Security Applications, Los Angeles, CA, USA.","DOI":"10.1145\/2991079.2991111"},{"key":"ref_11","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). Openai gym. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhou, S., Liu, J., Hou, D., Zhong, X., and Zhang, Y. (2021). Autonomous Penetration Testing Based on Improved Deep Q-Network. Appl. Sci., 11.","DOI":"10.3390\/app11198823"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Al Amin, M.A.R., Shetty, S., and Kamhoua, C. (2022, January 11\u201314). Cyber Deception Metrics for Interconnected Complex Systems. Proceedings of the 2022 Winter Simulation Conference (WSC), Singapore.","DOI":"10.1109\/WSC57314.2022.10015347"},{"key":"ref_14","first-page":"96","article-title":"Adversarial environment reinforcement learning algorithm for intrusion detection. Comput","volume":"159","author":"Caminero","year":"2019","journal-title":"Networks"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1186\/s42400-019-0027-x","article-title":"Adversarial attack and defense in reinforcement learning-from AI security view","volume":"2","author":"Chen","year":"2019","journal-title":"Cybersecurity"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Bhattacharya, A., Ramachandran, T., Banik, S., Dowling, C.P., and Bopardikar, S.D. (2020, January 9\u201310). Automated Adversary Emulation for Cyber-Physical Systems Via Reinforcement Learning. Proceedings of the 2020 IEEE International Conference on Intelligence and Security Informatics (ISI), Arlington, VA, USA.","DOI":"10.1109\/ISI49825.2020.9280521"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1016\/j.arcontrol.2022.01.001","article-title":"Reinforcement Learning for feedback-enabled cyber resilience","volume":"53","author":"Huang","year":"2022","journal-title":"Annu. Rev. Control"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Nguyen, T.T., and Reddi, V.J. (2021). Deep reinforcement learning for cyber security. IEEE Trans. Neural Netw. Learn. Syst.","DOI":"10.1109\/TNNLS.2021.3121870"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ganapathi, P., and Shanmugapriya, D. (2020). Handbook of Research on Machine and Deep Learning Applications for Cyber Security, IGI Global.","DOI":"10.4018\/978-1-5225-9611-0"},{"key":"ref_20","unstructured":"Standen, M., Lucas, M., Bowman, D., Richer, T.J., Kim, J., and Marriott, D. (2021). Cyborg: A gym for the development of au-tonomous cyber agents. arXiv."},{"key":"ref_21","unstructured":"Walter, E., Ferguson-Walter, K., and Ridley, A. (2021). Incorporating deception into cyberbattlesim for autonomous defense. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Wang, W., Sun, D., Jiang, F., Chen, X., and Zhu, C. (2022). Research and Challenges of Reinforcement Learning in Cyber Defense Decision-Making for Intranet Security. Algorithms, 15.","DOI":"10.3390\/a15040134"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Elderman, R., Pater, L.J.J., Thie, A.S., Drugan, M.M., and Wiering, M.M. (2017;, January 24\u201316). Adversarial Reinforcement Learning in a Cyber Security Simulation. Proceedings of the 9th International Conference on Agents and Artificial Intelligence, Porto, Portugal, 559\u2013566.","DOI":"10.5220\/0006197105590566"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Applebaum, A., Dennler, C., Dwyer, P., Moskowitz, M., Nguyen, H., Nichols, N., and Wolk, M. (2022, January 11). Bridging Automated to Autonomous Cyber Defense: Foundational Analysis of Tabular Q-Learning. Proceedings of the 15th ACM Workshop on Artificial Intelligence and Security, Los Angeles, CA, USA.","DOI":"10.1145\/3560830.3563732"},{"key":"ref_25","unstructured":"Seifert, C., Betser, M., Blum, W., Bono, J., Farris, K., Goren, E., Grana, J., Holsheimer, K., Marken, B., and Neil, J. (2021). Microsoft CyberBattleSim, Defender Research Team."},{"key":"ref_26","unstructured":"Sutton, R.S., and Barto, A.G. (2017). Reinforcement Learning: An Introduction, The MIT Press."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"e12905","DOI":"10.2196\/12905","article-title":"A Reinforcement Learning\u2013Based Method for Management of Type 1 Diabetes: Exploratory Study","volume":"4","author":"Javad","year":"2019","journal-title":"JMIR Diabetes"},{"key":"ref_28","unstructured":"Hjerde, S.T.N. (2020). Evaluating Deep Q-Learning Techniques for Controlling Type 1 Diabetes. [Master\u2019s Thesis, UiT Norges Arktiske Universitet]."},{"key":"ref_29","unstructured":"Nair, A., Srinivasan, P., Blackwell, S., Alcicek, C., Fearon, R., De Maria, A., and Silver, D. (2015). Massively parallel methods for deep reinforcement learning. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"352","DOI":"10.1016\/j.enbuild.2013.06.007","article-title":"Energy analysis of a building using artificial neural network: A review","volume":"65","author":"Kumar","year":"2013","journal-title":"Energy Build."},{"key":"ref_31","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1080\/23744731.2019.1680234","article-title":"Application of deep Q-networks for model-free optimal control balancing between different HVAC systems","volume":"26","author":"Ahn","year":"2019","journal-title":"Sci. Technol. Built Environ."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/6\/3000\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:51:47Z","timestamp":1760122307000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/6\/3000"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,10]]},"references-count":33,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["s23063000"],"URL":"https:\/\/doi.org\/10.3390\/s23063000","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,10]]}}}