{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,5]],"date-time":"2026-05-05T19:13:38Z","timestamp":1778008418925,"version":"3.51.4"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2025,1,10]],"date-time":"2025-01-10T00:00:00Z","timestamp":1736467200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Army Research Laboratory Cooperative","award":["W911NF-21-2-0284"],"award-info":[{"award-number":["W911NF-21-2-0284"]}]},{"name":"National Science Foundation","award":["#1850849 and #2051186"],"award-info":[{"award-number":["#1850849 and #2051186"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Cyber-Phys. Syst."],"published-print":{"date-parts":[[2025,1,31]]},"abstract":"<jats:p>Cyber resilience is the ability of a system to resist and recover from a cyber attack, thereby restoring the system\u2019s functionality. Effective design and development of a cyber resilient system requires experimental methods and tools for quantitative measuring of cyber resilience. This article describes an experimental method and test bed for obtaining resilience-relevant data as a system (in our case\u2014a truck) traverses its route, in repeatable, systematic experiments. We model a truck equipped with an autonomous cyber-defense system and which also includes inherent physical resilience features. When attacked by malware, this ensemble of cyber-physical features (i.e., \u201cbonware\u201d) strives to resist and recover from the performance degradation caused by the malware\u2019s attack. We propose parsimonious mathematical models to aid in quantifying systems\u2019 resilience to cyber attacks. Using the models, we identify quantitative characteristics obtainable from experimental data and show that these characteristics can serve as useful quantitative measures of cyber resilience.<\/jats:p>","DOI":"10.1145\/3703159","type":"journal-article","created":{"date-parts":[[2024,12,9]],"date-time":"2024-12-09T09:58:40Z","timestamp":1733738320000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Quantitative Measurement of Cyber Resilience: Modeling and Experimentation"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4918-5571","authenticated-orcid":false,"given":"Michael J.","family":"Weisman","sequence":"first","affiliation":[{"name":"DEVCOM Army Research Laboratory, Adelphi, Maryland, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1147-9726","authenticated-orcid":false,"given":"Alexander","family":"Kott","sequence":"additional","affiliation":[{"name":"DEVCOM Army Research Laboratory, Adelphi, Maryland, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-2526-5134","authenticated-orcid":false,"given":"Jason E.","family":"Ellis","sequence":"additional","affiliation":[{"name":"DEVCOM Army Research Laboratory, Adelphi, Maryland, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-3998-4798","authenticated-orcid":false,"given":"Brian J.","family":"Murphy","sequence":"additional","affiliation":[{"name":"Pennsylvania State University, University Park, Pennsylvania, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-7307-8808","authenticated-orcid":false,"given":"Travis W.","family":"Parker","sequence":"additional","affiliation":[{"name":"ICF International Inc, Reston, Virginia, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1398-307X","authenticated-orcid":false,"given":"Sidney","family":"Smith","sequence":"additional","affiliation":[{"name":"DEVCOM Army Research Laboratory, Adelphi, Maryland, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2600-5937","authenticated-orcid":false,"given":"Joachim","family":"Vandekerckhove","sequence":"additional","affiliation":[{"name":"University of California, Irvine, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,1,10]]},"reference":[{"key":"e_1_3_4_2_1","unstructured":"A. Alexeev D. S. Henshel K. Levitt P. McDaniel B. Rivera S. Templeton and M. Weisman. 2017. Constructing a science of cyber-resilience for military systems. In NATO IST-153 Workshop on Cyber Resilience 23\u201325."},{"key":"e_1_3_4_3_1","unstructured":"P. Beling B. Horowitz and T. McDermott. 2021. Developmental Test and Evaluation (DTE & A) and Cyberattack Resilient Systems. Technical Report Serc-2021-tr-015. Systems Engineering Research Center Hoboken NJ."},{"key":"e_1_3_4_4_1","doi-asserted-by":"crossref","unstructured":"M. Bozdal M. Randa M. Samie and I. Jennions. 2018a. Hardware trojan enabled denial of service attack on CAN bus. Procedia Manufacturing 16 (2018) 47\u201352.","DOI":"10.1016\/j.promfg.2018.10.158"},{"key":"e_1_3_4_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/iCCECOME.2018.8658720"},{"key":"e_1_3_4_6_1","doi-asserted-by":"crossref","unstructured":"J. Daily R. Gamble S. Moffitt C. Raines P. Harris J. Miran I. Ray S. Mukherjee H. Shirazi and J. Johnson. 2016. Towards a cyber assurance testbed for heavy vehicle electronic controls. SAE International Journal of Commercial Vehicles 9 2 (2016) 339\u2013349.","DOI":"10.4271\/2016-01-8142"},{"key":"e_1_3_4_7_1","doi-asserted-by":"crossref","unstructured":"J. H. Dillon. 1947. Resilience of fibers and fabrics. Textile Research Journal 17 4 (1947) 207\u2013213.","DOI":"10.1177\/004051754701700403"},{"key":"e_1_3_4_8_1","doi-asserted-by":"crossref","unstructured":"B. Dupont. 2019. The cyber-resilience of financial institutions: Significance and applicability. Journal of Cybersecurity 5 1 (2019) tyz013.","DOI":"10.1093\/cybsec\/tyz013"},{"key":"e_1_3_4_9_1","first-page":"841","volume-title":"IEEE Military Communications Conference","author":"Ellis J.","year":"2022","unstructured":"J. Ellis, T. Parker, J. Vandekerckhove, B. Murphy, S. Smith, A. Kott, and M. Weisman. 2022. Experimental infrastructure for study of measurements of resilience. In IEEE Military Communications Conference, 841\u2013846."},{"key":"e_1_3_4_10_1","doi-asserted-by":"crossref","unstructured":"C. Folke S. R. Carpenter B. Walker M. Scheffer T. Chapin and J. Rockstr\u00f6m. 2010. Resilience thinking: Integrating resilience adaptability and transformability. Ecology and Society 15 4 (2010). Retrieved from https:\/\/www.jstor.org\/stable\/26268226?seq=6","DOI":"10.5751\/ES-03610-150420"},{"key":"e_1_3_4_11_1","volume-title":"9th USENIX Workshop on Offensive Technologies (WOOT \u201915)","author":"Foster I.","year":"2015","unstructured":"I. Foster, A. Prudhomme, K. Koscher, and S. Savage. 2015. Fast and vulnerable: A story of telematic failures. In 9th USENIX Workshop on Offensive Technologies (WOOT \u201915). USENIX Association, Washington, D.C. Retrieved from https:\/\/www.usenix.org\/conference\/woot15\/workshop-program\/presentation\/foster"},{"key":"e_1_3_4_12_1","doi-asserted-by":"crossref","unstructured":"A. Gelman J. B. Carlin H. S. Stern and D. B. Rubin. 1995. Bayesian Data Analysis. Chapman and Hall\/CRC.","DOI":"10.1201\/9780429258411"},{"key":"e_1_3_4_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.iot.2020.100204"},{"key":"e_1_3_4_14_1","volume-title":"Fifth World Conference on Risk","author":"Henshel D. S.","year":"2019","unstructured":"D. S. Henshel, K. Levitt, S. Templeton, M. G. Cains, A. Alexeev, B. Blakely, P. McDaniel, G. Wehner, J. Rowell, and M. Weisman. 2019. The science of cyber resilience: Characteristics and initial system taxonomy. In Fifth World Conference on Risk."},{"key":"e_1_3_4_15_1","doi-asserted-by":"crossref","unstructured":"L. Herrington and R. Aldrich. 2013. The future of cyber-resilience in an age of global complexity. Politics 33 4 (2013) 299\u2013310.","DOI":"10.1111\/1467-9256.12035"},{"key":"e_1_3_4_16_1","doi-asserted-by":"crossref","unstructured":"H. Herrman D. E. Stewart N. Diaz-Granados E. L. Berger B. Jackson and T. Yuen. 2011. What is resilience? The Canadian Journal of Psychiatry 56 5 (2011) 258\u2013265.","DOI":"10.1177\/070674371105600504"},{"key":"e_1_3_4_17_1","doi-asserted-by":"crossref","unstructured":"R. M. Hoffman. 1948. A generalized concept of resilience. Textile Research Journal 18 3 (1948) 141\u2013148.","DOI":"10.1177\/004051754801800301"},{"key":"e_1_3_4_18_1","first-page":"1","volume-title":"2nd Workshop on Embedded Systems Security (WESS \u201907)","author":"Hoppe T.","year":"2007","unstructured":"T. Hoppe and J. Dittman. 2007. Sniffing\/replay attacks on CAN buses: A simulated attack on the electric window lift classified using an adapted CERT taxonomy. In 2nd Workshop on Embedded Systems Security (WESS \u201907), 1\u20136."},{"key":"e_1_3_4_19_1","doi-asserted-by":"crossref","unstructured":"S. Hosseini K. Barker and J. E. Ramirez-Marquez. 2016. A review of definitions and measures of system resilience. Reliability Engineering & System Safety 145 (2016) 47\u201361.","DOI":"10.1016\/j.ress.2015.08.006"},{"key":"e_1_3_4_20_1","doi-asserted-by":"crossref","unstructured":"Y. Huang L. Huang and Q. Zhu. 2022. Reinforcement learning for feedback-enabled cyber resilience. Annual Reviews in Control 53 (2022) 273\u2013295.","DOI":"10.1016\/j.arcontrol.2022.01.001"},{"key":"e_1_3_4_21_1","doi-asserted-by":"publisher","DOI":"10.22215\/timreview\/889"},{"key":"e_1_3_4_22_1","unstructured":"Keysight Technologies. 2021. Open source in test automation. Retrieved from https:\/\/opentap.io\/assets\/Open-Source-in-Test-Automation-v3.pdf\/"},{"key":"e_1_3_4_23_1","doi-asserted-by":"crossref","unstructured":"K. Koscher A. Czeskis F. Roesner S. Patel T. Kohno S. Checkoway D. McCoy B. Kantor D. Anderson H Shacham et al. 2010. Experimental security analysis of a modern automobile. In 2010 IEEE Symposium on Security and Privacy. IEEE 447\u2013462.","DOI":"10.1109\/SP.2010.34"},{"key":"e_1_3_4_24_1","doi-asserted-by":"crossref","unstructured":"A. Kott M. S. Golan B. D. Trump and I. Linkov. 2021. Cyber resilience: By design or by intervention? Computer 54 8 (2021) 112\u2013117.","DOI":"10.1109\/MC.2021.3082836"},{"key":"e_1_3_4_25_1","doi-asserted-by":"crossref","unstructured":"A. Kott and I. Linkov. 2019. Cyber Resilience of Systems and Networks. Springer International Publishing New York NY.","DOI":"10.1007\/978-3-319-77492-3"},{"key":"e_1_3_4_26_1","doi-asserted-by":"crossref","unstructured":"A. Kott and I. Linkov. 2021. To improve cyber resilience measure it. Computer 54 2 (Feb. 2021) 80\u201385.","DOI":"10.1109\/MC.2020.3038411"},{"key":"e_1_3_4_27_1","doi-asserted-by":"crossref","unstructured":"A. Kott and P. Theron. 2020. Doers not watchers: Intelligent autonomous agents are a path to cyber resilience. IEEE Security & Privacy 18 3 (2020) 62\u201366.","DOI":"10.1109\/MSEC.2020.2983714"},{"key":"e_1_3_4_28_1","unstructured":"A. Kott P. Th\u00e9ron M. Dra\u0161ar E. Dushku B. LeBlanc P. Losiewicz A. Guarino L. V. Mancini A. Panico M. Pihelgas et al. 2018. Autonomous intelligent cyber-defense agent (AICE) reference architecture. arXiv:1803.10664. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.1803.10664"},{"key":"e_1_3_4_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/MILCOM55135.2022.10017731"},{"key":"e_1_3_4_30_1","doi-asserted-by":"crossref","unstructured":"A. Kott M. Weisman J. Vandekerckhove J. Ellis T. Parker B. Murphy and S. Smith. 2023. A Methodology for Quantitative Measurement of Cyber Resilience (QMOCR). Technical Report ARL-TR-9672.","DOI":"10.21236\/AD1210026"},{"key":"e_1_3_4_31_1","doi-asserted-by":"crossref","unstructured":"A. K. Ligo A. Kott and I. Linkov. 2021. How to measure cyber-resilience of a system with autonomous agents: Approaches and challenges. IEEE Engineering Management Review 49 2 (2021) 89\u201397.","DOI":"10.1109\/EMR.2021.3074288"},{"key":"e_1_3_4_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/.2005.1466986"},{"key":"e_1_3_4_33_1","doi-asserted-by":"crossref","unstructured":"I. Linkov D. A. Eisenberg K. Plourde T. P. Seager J. Allen and A. Kott. 2013. Resilience metrics for cyber systems. Environment Systems and Decisions 33 4 (2013) 471\u2013476.","DOI":"10.1007\/s10669-013-9485-y"},{"key":"e_1_3_4_34_1","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-018-02567-0"},{"key":"e_1_3_4_35_1","doi-asserted-by":"publisher","unstructured":"D. Matzke U. Boehm and J. Vandekerckhove. 2017. Bayesian inference for psychology part III: Parameter estimation in nonstandard models. Psychonomic Bulletin & Review 25 1 (Nov. 2017) 77\u2013101. DOI: 10.3758\/s13423-017-1394-5","DOI":"10.3758\/s13423-017-1394-5"},{"key":"e_1_3_4_36_1","unstructured":"T Miasko. 2017. pyjags (version 1.3.8) [computer software]. Retrieved from https:\/\/github.com\/tmiasko\/pyjags"},{"key":"e_1_3_4_37_1","first-page":"260","article-title":"Adventures in automotive networks and control units","volume":"21","author":"Miller C.","year":"2013","unstructured":"C. Miller and C. Valasek. 2013. Adventures in automotive networks and control units. Def Con 21, 260\u2013264 (2013), 15\u201331.","journal-title":"Def Con"},{"key":"e_1_3_4_38_1","unstructured":"C. Miller and C. Valasek. 2015. Remote exploitation of an unaltered passenger vehicle. In Black Hat USA."},{"key":"e_1_3_4_39_1","doi-asserted-by":"publisher","DOI":"10.1243\/0954407042580110"},{"key":"e_1_3_4_40_1","unstructured":"H. Shikata T. Yamashita K. Arai T. Nakano K. Hatanaka and H. Fujikawa. 2019. Digital twin environment to integrate vehicle simulation and physical verification. SEI Technical Review 88 (2019) 18\u201321."},{"key":"e_1_3_4_41_1","doi-asserted-by":"publisher","DOI":"10.21236\/AD1205369"},{"key":"e_1_3_4_42_1","doi-asserted-by":"crossref","unstructured":"R. A. Struble. 1962. Nonlinear Differential Equations. McGraw-Hill.","DOI":"10.1063\/1.3058323"},{"key":"e_1_3_4_43_1","doi-asserted-by":"crossref","unstructured":"G. Teschl. 2012. Ordinary Differential Equations and Dynamical Systems. American Mathematical Society.","DOI":"10.1090\/gsm\/140"},{"key":"e_1_3_4_44_1","unstructured":"T. Toyama T. Yoshida H. Oguma and T. Matsumoto. 2018. PASTA: Portable automotive security testbed with adaptability. In Black Hat Europe. Retrieved from https:\/\/i.blackhat.com\/eu-18\/Wed-Dec-5\/eu-18-Toyama-PASTA-Portable-Automotive-Security-Testbed-with-Adaptability.pdf"},{"key":"e_1_3_4_45_1","unstructured":"Unity Technologies. 2021. The leading platform for creating interactive real-time content. Retrieved August 2021 from https:\/\/unity.com\/ (v2020.3.18f1)"},{"key":"e_1_3_4_46_1","doi-asserted-by":"publisher","unstructured":"Q. Wang Y. Qian Z. Lu Y. Shoukry and G. Qu. 2018. A delay based plug-in-monitor for intrusion detection in controller area network. In 2018 Asian Hardware Oriented Security and Trust Symposium (AsianHOST \u201918) 86\u201391. DOI: 10.1109\/AsianHOST.2018.8607178","DOI":"10.1109\/AsianHOST.2018.8607178"},{"key":"e_1_3_4_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/CISS56502.2023.10089725"},{"key":"e_1_3_4_48_1","doi-asserted-by":"publisher","DOI":"10.3389\/fnbeh.2013.00010"},{"key":"e_1_3_4_49_1","doi-asserted-by":"crossref","unstructured":"B. Zou P. Choobchian and J. Rozenberg. 2021. Cyber resilience of autonomous mobility systems: Cyber-attacks and resilience-enhancing strategies. Journal of Transportation Security 14 3 (2021) 1\u201319.","DOI":"10.1007\/s12198-021-00230-w"}],"container-title":["ACM Transactions on Cyber-Physical Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3703159","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3703159","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3703159","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,5]],"date-time":"2026-05-05T19:03:20Z","timestamp":1778007800000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3703159"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,10]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1,31]]}},"alternative-id":["10.1145\/3703159"],"URL":"https:\/\/doi.org\/10.1145\/3703159","relation":{},"ISSN":["2378-962X","2378-9638"],"issn-type":[{"value":"2378-962X","type":"print"},{"value":"2378-9638","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,10]]},"assertion":[{"value":"2023-03-30","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}