{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:40:03Z","timestamp":1760146803695,"version":"build-2065373602"},"reference-count":24,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2024,12,9]],"date-time":"2024-12-09T00:00:00Z","timestamp":1733702400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>As network systems become larger and more complex, there is an increasing focus on how to verify the security of systems that are at risk of being attacked. Automated penetration testing is one of the effective ways to achieve this. Uncertainty caused by adversarial relationships and the \u201cfog of war\u201d is an unavoidable problem in penetration testing research. However, related methods have largely focused on the uncertainty of state transitions in the penetration testing process, and have generally ignored the uncertainty caused by partially observable conditions. To address this new uncertainty introduced by partially observable conditions, we model the penetration testing process as a partially observable Markov decision process (POMDP) and propose an intelligent penetration testing decision method compatible with it. We experimentally validate the impact of partially observable conditions on penetration testing. The experimental results show that our method can effectively mitigate the negative impact of partially observable conditions on penetration testing decision. It also exhibits good scalability as the size of the target network increases.<\/jats:p>","DOI":"10.3390\/systems12120546","type":"journal-article","created":{"date-parts":[[2024,12,9]],"date-time":"2024-12-09T10:11:47Z","timestamp":1733739107000},"page":"546","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Modelling and Intelligent Decision of Partially Observable Penetration Testing for System Security Verification"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0666-4102","authenticated-orcid":false,"given":"Xiaojian","family":"Liu","sequence":"first","affiliation":[{"name":"College of Computer Science, Beijing University of Technology, Beijing 100124, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yangyang","family":"Zhang","sequence":"additional","affiliation":[{"name":"China Electronics Standardization Institute, Beijing 100007, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenpeng","family":"Li","sequence":"additional","affiliation":[{"name":"China Electronics Standardization Institute, Beijing 100007, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wen","family":"Gu","sequence":"additional","affiliation":[{"name":"College of Computer Science, Beijing University of Technology, Beijing 100124, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,12,9]]},"reference":[{"doi-asserted-by":"crossref","unstructured":"Alhamed, M., and Rahman, M.M.H. (2023). A Systematic Literature Review on Penetration Testing in Networks: Future Research Directions. Appl. Sci., 13.","key":"ref_1","DOI":"10.3390\/app13126986"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2130014","DOI":"10.1080\/17517575.2022.2130014","article-title":"AI-enabled IoT penetration testing: State-of-the-art and research challenges","volume":"17","author":"Greco","year":"2023","journal-title":"Enterp. Inf. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1007\/s10844-022-00738-0","article-title":"Hierarchical reinforcement learning for efficient and effective automated penetration testing of large networks","volume":"60","author":"Ghanem","year":"2023","journal-title":"J. Intell. Inf. Syst."},{"key":"ref_4","first-page":"2095","article-title":"Domain-Independent Intelligent Planning Technology and Its Applicaion to Automated Penetration Testing Oriented Attack Path Discovery","volume":"42","author":"Zhang","year":"2020","journal-title":"J. Electron. Inf. Technol."},{"key":"ref_5","first-page":"679","article-title":"A Markovian decision process","volume":"6","author":"Bellman","year":"1957","journal-title":"J. Math. Mech."},{"doi-asserted-by":"crossref","unstructured":"Spaan, M.T.J. (2012). Partially observable Markov decision processes. Reinforcement Learning: State-of-the-Art, Springer.","key":"ref_6","DOI":"10.1007\/978-3-642-27645-3_12"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MSP.2017.2743240","article-title":"Deep reinforcement learning: A brief survey","volume":"34","author":"Arulkumaran","year":"2017","journal-title":"IEEE Signal Process. Mag."},{"unstructured":"Ghojogh, B., and Ghodsi, A. (2023). Recurrent neural networks and long short-term memory networks: Tutorial and survey. arXiv.","key":"ref_8"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"5834434","DOI":"10.1155\/2023\/5834434","article-title":"DQfD-AIPT: An Intelligent Penetration Testing Framework Incorporating Expert Demonstration Data","volume":"2023","author":"Wang","year":"2023","journal-title":"Secur. Commun. Netw."},{"unstructured":"Schwartz, J., and Kurniawati, H. (2019). Autonomous penetration testing using reinforcement learning. arXiv.","key":"ref_10"},{"key":"ref_11","first-page":"40","article-title":"Intelligent Penetration Testing Path Discovery Based on Deep Reinforcement Learning","volume":"48","author":"Zhou","year":"2021","journal-title":"Comput. Sci."},{"doi-asserted-by":"crossref","unstructured":"Nguyen, H.V., Teerakanok, S., Inomata, A., and Uehara, T. (2021, January 11\u201313). The Proposal of Double Agent Architecture using Actor-critic Algorithm for Penetration Testing. Proceedings of the 7th International Conference on Information Systems Security and Privacy, Vienna, Austria.","key":"ref_12","DOI":"10.5220\/0010232504400449"},{"key":"ref_13","first-page":"308","article-title":"Intelligent Attack Path Discovery Based on Hierarchical Reinforcement Learning","volume":"50","author":"Zeng","year":"2023","journal-title":"Comput. Sci."},{"doi-asserted-by":"crossref","unstructured":"Schwartz, J., Kurniawati, H., and El-Mahassni, E. (2020, January 14\u201319). Pomdp+ information-decay: Incorporating defender\u2019s behaviour in autonomous penetration testing. Proceedings of the International Conference on Automated Planning and Scheduling, Nancy, France.","key":"ref_14","DOI":"10.1609\/icaps.v30i1.6666"},{"unstructured":"Sarraute, C., Buffet, O., and Hoffmann, J. (2013). Penetration testing== POMDP solving?. arXiv.","key":"ref_15"},{"unstructured":"Shmaryahu, D., Shani, G., Hoffmann, J., and Steinmetz, M. (2017, January 20). Partially observable con-tingent planning for penetration testing. Proceedings of the Iwaise: First International Workshop on Artificial Intelligence in Security, Melbourne, Australia.","key":"ref_16"},{"doi-asserted-by":"crossref","unstructured":"Mairh, A., Barik, D., Verma, K., and Jena, D. (2011, January 12\u201314). Honeypot in network security: A survey. Proceedings of the 2011 International Conference on Communication, Computing & Security, Odisha, India.","key":"ref_17","DOI":"10.1145\/1947940.1948065"},{"unstructured":"(2024, November 28). Common Vulnerabilities and Exploits. Available online: https:\/\/cve.mitre.org\/.","key":"ref_18"},{"unstructured":"(2024, November 28). Common Attack Pattern Enumeration and Classification. Available online: https:\/\/capec.mitre.org\/.","key":"ref_19"},{"unstructured":"(2024, November 28). Common Vulnerability Scoring System. Available online: https:\/\/www.first.org\/cvss\/v4.0\/specification-document.","key":"ref_20"},{"unstructured":"Backes, M., Hoffmann, J., Kunnemann, R., Speicher, P., and Steinmetz, M. (2017). Simulated Penetration Testing and Mitigation Analysis. arXiv.","key":"ref_21"},{"unstructured":"Rabinowitz, N., Perbet, F., Song, F., Zhang, C., Eslami, S.M.A., and Botvinick, M. (2018, January 10\u201315). Machine theory of mind. Proceedings of the International Conference on Machine Learning. PMLR, Stockholm, Sweden.","key":"ref_22"},{"doi-asserted-by":"crossref","unstructured":"Chowdhary, A., Huang, D., Mahendran, J.S., Romo, D., Deng, Y., and Sabur, A. (2020, January 17\u201319). Autonomous security analysis and penetration testing. Proceedings of the 2020 16th International Conference on Mobility, Sensing and Networking (MSN), Tokyo, Japan.","key":"ref_23","DOI":"10.1109\/MSN50589.2020.00086"},{"doi-asserted-by":"crossref","unstructured":"Zhou, S., Liu, J., Hou, D., Zhong, X., and Zhang, Y. (2021). Autonomous penetration testing based on improved deep q-network. Appl. Sci., 11.","key":"ref_24","DOI":"10.3390\/app11198823"}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/12\/12\/546\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:50:06Z","timestamp":1760115006000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/12\/12\/546"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,9]]},"references-count":24,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["systems12120546"],"URL":"https:\/\/doi.org\/10.3390\/systems12120546","relation":{},"ISSN":["2079-8954"],"issn-type":[{"type":"electronic","value":"2079-8954"}],"subject":[],"published":{"date-parts":[[2024,12,9]]}}}