{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T16:21:18Z","timestamp":1778257278305,"version":"3.51.4"},"reference-count":78,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T00:00:00Z","timestamp":1689206400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Mathematics for Artificial Reasoning in Science"},{"name":"Pacific Northwest National Laboratory, USA"},{"name":"NSF","award":["CNS-2134076"],"award-info":[{"award-number":["CNS-2134076"]}]},{"name":"Secure and Trustworthy Cyberspace"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Cyber-Phys. Syst."],"published-print":{"date-parts":[[2023,7,31]]},"abstract":"<jats:p>\n            Security of cyber-physical systems (CPS) continues to pose new challenges due to the tight integration and operational complexity of the cyber and physical components. To address these challenges, this article presents a domain-aware, optimization-based approach to determine an effective defense strategy for CPS in an automated fashion\u2014by emulating a strategic adversary in the loop that exploits system vulnerabilities, interconnection of the CPS, and the dynamics of the physical components. Our approach builds on an adversarial decision-making model based on a Markov Decision Process (MDP) that determines the optimal cyber (discrete) and physical (continuous) attack actions over a CPS attack graph. The defense planning problem is modeled as a non-zero-sum game between the adversary and defender. We use a model-free reinforcement learning method to solve the adversary\u2019s problem as a function of the defense strategy. We then employ Bayesian optimization (BO) to find an approximate\n            <jats:italic>best-response<\/jats:italic>\n            for the defender to harden the network against the resulting adversary policy. This process is iterated multiple times to improve the strategy for both players. We demonstrate the effectiveness of our approach on a ransomware-inspired graph with a smart building system as the physical process. Numerical studies show that our method converges to a Nash equilibrium for various defender-specific costs of network hardening.\n          <\/jats:p>","DOI":"10.1145\/3596222","type":"journal-article","created":{"date-parts":[[2023,5,18]],"date-time":"2023-05-18T12:18:13Z","timestamp":1684412293000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Automated Adversary-in-the-Loop Cyber-Physical Defense Planning"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9949-7173","authenticated-orcid":false,"given":"Sandeep","family":"Banik","sequence":"first","affiliation":[{"name":"Michigan State University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5705-7863","authenticated-orcid":false,"given":"Thiagarajan","family":"Ramachandran","sequence":"additional","affiliation":[{"name":"Pacific Northwest National Laboratory, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3727-350X","authenticated-orcid":false,"given":"Arnab","family":"Bhattacharya","sequence":"additional","affiliation":[{"name":"Pacific Northwest National Laboratory, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0813-7867","authenticated-orcid":false,"given":"Shaunak D.","family":"Bopardikar","sequence":"additional","affiliation":[{"name":"Michigan State University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,7,13]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"R 2021 MITRE ATT&CK"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISGT49243.2021.9372209"},{"key":"e_1_3_2_4_2","unstructured":"Abdullah Al-Dujaili Erik Hemberg and Una-May O\u2019Reilly. 2018. Approximating nash equilibria for black-box games: A bayesian optimization approach. International Workshop on Optimization in Multiagent Systems AAMAS. https:\/\/web.ecs.syr.edu\/ffiorett\/cfp\/OPTMAS18\/papers\/paper_14.pdf."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/CNS48642.2020.9162207"},{"key":"e_1_3_2_6_2","article-title":"MITRE ATT&CK\u00ae for industrial control systems: Design and philosophy","author":"Alexander Otis","year":"2020","unstructured":"Otis Alexander, Misha Belisle, and Jacob Steele. 2020. MITRE ATT&CK\u00ae for industrial control systems: Design and philosophy. The MITRE Corporation: Bedford, MA, USA (2020), 29. https:\/\/attack.mitre.org\/docs\/ATTACK_for_ICS_Philosophy_March_2020.pdf.","journal-title":"The MITRE Corporation: Bedford, MA, USA"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/586110.586140"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1093\/comjnl\/bxz146"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1002\/qre.864"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSCON.2018.8369518"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611971132"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10898-009-9496-x"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1287\/ijoc.1090.0319"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1287\/opre.1090.0715"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISI49825.2020.9280521"},{"key":"e_1_3_2_16_2","volume-title":"Incremental Dynamic Programming for On-Line Adaptive Optimal Control","author":"Bradtke Steven J.","year":"1994","unstructured":"Steven J. Bradtke. 1994. Incremental Dynamic Programming for On-Line Adaptive Optimal Control. Ph.D. Dissertation. Citeseer."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMST.2015.2494502"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.5555\/1137817"},{"key":"e_1_3_2_19_2","unstructured":"Somali Chaterji Parinaz Naghizadeh Muhammad Ashraful Alam Saurabh Bagchi Mung Chiang David Corman Brian Henz Suman Jana Na Li Shaoshuai Mou Meeko Oishi Chunyi Peng Tiark Rompf Ashutosh Sabharwal Shreyas Sundaram James Weimer and Jennifer Weller. 2019. Resilient Cyberphysical Systems and their Application Drivers: A Technology Roadmap. arXiv:2001.00090. Retrieved from https:\/\/arxiv.org\/abs\/2001.00090."},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSG.2011.2160000"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSG.2018.2790704"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/VTCSpring.2018.8417695"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3474718.3474722"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00993978"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.arcontrol.2019.04.011"},{"key":"e_1_3_2_26_2","volume-title":"Cybersecurity in Building Automation Systems","author":"Santos Daniel dos","year":"2019","unstructured":"Daniel dos Santos, Clement Speybrouck, and Elisa Costante. 2019. Cybersecurity in Building Automation Systems. Technical Report. Forescout Technologies."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.5555\/2832249.2832322"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v28i1.8843"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1006\/game.1999.0738"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1002\/qre.4680090306"},{"issue":"2","key":"e_1_3_2_31_2","first-page":"97","article-title":"Approximation to bayes risk in repeated play","volume":"3","author":"Hannan James","year":"1957","unstructured":"James Hannan. 1957. Approximation to bayes risk in repeated play. Contributions to the Theory of Games 3, 2 (1957), 97\u2013139.","journal-title":"Contributions to the Theory of Games"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISRCS.2012.6309311"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-75268-6_7"},{"key":"e_1_3_2_34_2","doi-asserted-by":"crossref","unstructured":"Mariam Ibrahim and Ahmad Alsheikh. 2019. Automatic hybrid attack graph (AHAG) generation for complex engineering systems. Processes 7 11 (2019). https:\/\/www.mdpi.com\/2227-9717\/7\/11\/787.","DOI":"10.3390\/pr7110787"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00186-014-0491-8"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10586-017-1117-8"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cosrev.2019.100219"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2011.67"},{"key":"e_1_3_2_39_2","first-page":"62","article-title":"German steel mill cyber attack","volume":"30","author":"Lee Robert M.","year":"2014","unstructured":"Robert M. Lee, Michael J. Assante, and Tim Conway. 2014. German steel mill cyber attack. Industrial Control Systems 30 (2014), 62.","journal-title":"Industrial Control Systems"},{"key":"e_1_3_2_40_2","article-title":"Polish teen derails tram after hacking train network","volume":"11","author":"Leyden John","year":"2008","unstructured":"John Leyden. 2008. Polish teen derails tram after hacking train network. The Register 11 (2008).","journal-title":"The Register"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1201\/9781351006620"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-015-2958-x"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/2179298.2179368"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/GLOCOM.2017.8253987"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2018.12.006"},{"key":"e_1_3_2_46_2","first-page":"72","volume-title":"2019 Proceedings of the Cyber-Physical Systems PhD Workshop","author":"Macas Mayra","year":"2013","unstructured":"Mayra Macas and Wu Chunming. 2013. Enhanced cyber-physical security through deep learning techniques. In 2019 Proceedings of the Cyber-Physical Systems PhD Workshop. 72\u201383. Retrieved from http:\/\/ceur-ws.org\/Vol-2457\/8.pdf."},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/366173.366183"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCNS.2016.2573039"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/CCTA41146.2020.9206253"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/2808475.2808482"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-30719-6_2"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1049\/cps2.12009"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2021.3121870"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2018.2885530"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMST.2020.3036778"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/DMC51747.2021.9529953"},{"key":"e_1_3_2_57_2","series-title":"Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation\u2014Volume 1","first-page":"525","author":"Pelikan Martin","year":"1999","unstructured":"Martin Pelikan, David E. Goldberg, and Erick Cant\u00fa-Paz. 1999. BOA: The bayesian optimization algorithm. In Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation\u2014Volume 1 (Orlando, Florida) (GECCO\u201999). Morgan Kaufmann Publishers Inc., San Francisco, CA, 525\u2013532."},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.enbuild.2017.07.027"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10898-018-0688-0"},{"key":"e_1_3_2_60_2","unstructured":"PNNL. 2019. Python Systems Library. Retrieved from https:\/\/github.com\/pnnl\/psl."},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1049\/et.2016.0116"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00942191"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2017.298"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/THS.2016.7568884"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-32430-8_25"},{"key":"e_1_3_2_66_2","unstructured":"Dinuka Sahabandu Shana Moothedath Joey Allen Linda Bushnell Wenke Lee and Radha Poovendran. 2021. A Reinforcement Learning Approach for Dynamic Information Flow Tracking Games for Detecting Advanced Persistent Threats. arXiv:2007.00076. Retrieved from https:\/\/arxiv.org\/abs\/2007.00076."},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2018.8619416"},{"key":"e_1_3_2_68_2","unstructured":"Aaron Schlenker Omkar Thakoor Haifeng Xu Fei Fang Milind Tambe Long Tran-Thanh Phebe Vayanos and Yevgeniy Vorobeychik. 2018. Deceiving cyber adversaries: A game theoretic approach(AAMAS\u201918). International Foundation for Autonomous Agents and Multiagent Systems Richland SC 892\u2013900."},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2011.2182033"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/1037187.1024404"},{"key":"e_1_3_2_71_2","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton Richard S.","year":"2018","unstructured":"Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. MIT Press."},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553501"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2805690"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/IECON.2018.8591773"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2016.2524258"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2013.08.037"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2016.2607701"},{"key":"e_1_3_2_78_2","first-page":"321","volume-title":"Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms","author":"Zhang Kaiqing","year":"2021","unstructured":"Kaiqing Zhang, Zhuoran Yang, and Tamer Ba\u015far. 2021. Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms. Springer, Cham, 321\u2013384."},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSG.2016.2523515"}],"container-title":["ACM Transactions on Cyber-Physical Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3596222","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3596222","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:47:58Z","timestamp":1750178878000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3596222"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,13]]},"references-count":78,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,7,31]]}},"alternative-id":["10.1145\/3596222"],"URL":"https:\/\/doi.org\/10.1145\/3596222","relation":{},"ISSN":["2378-962X","2378-9638"],"issn-type":[{"value":"2378-962X","type":"print"},{"value":"2378-9638","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,13]]},"assertion":[{"value":"2021-07-08","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-04-11","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-07-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}