{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T14:58:47Z","timestamp":1775573927463,"version":"3.50.1"},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"name":"Research Institute in Trustworthy Interconnected Cyber-physical Systems"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Digital Threats"],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>Operational technology (OT) is foundational to critical sectors such as water, energy and manufacturing, yet securing it presents unique challenges. Conventional IT penetration testing methods cannot be directly applied to OT because of the risks of disrupting live production environments. As a result, there is little consensus on how penetration testing should be conducted in OT environments, and practice remains inconsistent and poorly understood. This article explores the state of OT penetration testing through a review of existing literature and qualitative interviews with both practitioners and procurers. The findings highlight three central challenges: the absence of standardised methodologies, the limited relevance of IT-style vulnerability reporting, and cultural tensions between cyber security specialists and operational engineers. From these insights, the article proposes a set of short-, medium- and long-term recommendations to improve practice and points to key areas for future research.<\/jats:p>","DOI":"10.1145\/3802592","type":"journal-article","created":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T20:17:43Z","timestamp":1773778663000},"page":"1-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["The Challenges of Operational Technology Penetration Testing"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3902-5056","authenticated-orcid":false,"given":"Richard","family":"Derbyshire","sequence":"first","affiliation":[{"name":"Orange Cyberdefense, London, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,4,7]]},"reference":[{"key":"e_1_3_1_2_2","volume-title":"Industrial Cybersecurity","author":"Ackerman Pascal","year":"2021","unstructured":"Pascal Ackerman. 2021. Industrial Cybersecurity (2nd ed.). Packt Publishing.","edition":"2"},{"key":"e_1_3_1_3_2","doi-asserted-by":"crossref","DOI":"10.4135\/9781849209335","volume-title":"Interviewing for Social Scientists: An Introductory Resource with Examples","author":"Arksey Hilary","year":"1999","unstructured":"Hilary Arksey and Peter T. Knight. 1999. Interviewing for Social Scientists: An Introductory Resource with Examples. SAGE Publications."},{"key":"e_1_3_1_4_2","unstructured":"Michael J. Assante and Robert M. Lee. 2015. The Industrial Control System Cyber Kill Chain. Technical Report. SANS Institute. Retrieved from https:\/\/www.sans.org\/white-papers\/36297\/Whitepaper"},{"key":"e_1_3_1_5_2","volume-title":"Hacking Exposed Industrial Control Systems: ICS and SCADA Security Secrets & Solutions","author":"Bodungen Clint","year":"2016","unstructured":"Clint Bodungen, Bryan L. Singer, Aaron Shbeeb, Kyle Wilhoit, and Stephen Hilt. 2016. Hacking Exposed Industrial Control Systems: ICS and SCADA Security Secrets & Solutions (1st ed.). McGraw-Hill Education.","edition":"1"},{"key":"e_1_3_1_6_2","volume-title":"Experimental and Quasi-Experimental Designs for Research","author":"Campbell Donald T.","year":"1963","unstructured":"Donald T. Campbell and Julian C. Stanley. 1963. Experimental and Quasi-Experimental Designs for Research. Rand McNally."},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3471621.3471864"},{"key":"e_1_3_1_8_2","unstructured":"Cisco Systems Inc. 2019. Networking and Security in Industrial Automation Environments. Retrieved September 9 2025 from https:\/\/www.cisco.com\/c\/en\/us\/td\/docs\/solutions\/Verticals\/Industrial_Automation\/IA_Horizontal\/DG\/Industrial-AutomationDG\/Industrial-AutomationDG.html"},{"key":"e_1_3_1_9_2","first-page":"93","volume-title":"Doing Qualitative Research","author":"Crabtree Benjamin F.","year":"1992","unstructured":"Benjamin F. Crabtree and William L. Miller. 1992. A template approach to text analysis: Developing and using codebooks. In Doing Qualitative Research, Benjamin F. Crabtree and William L. Miller (Eds.), SAGE Publications, 93\u2013109."},{"key":"e_1_3_1_10_2","unstructured":"CREST. 2022. A Guide for Running an Effective Penetration Testing Programme. Retrieved from https:\/\/www.crest-approved.org\/wp-content\/uploads\/2022\/04\/CREST-Penetration-Testing-Guide-1.pdf"},{"key":"e_1_3_1_11_2","unstructured":"Cybersecurity and Infrastructure Security Agency (CISA). 2026. Validated Architecture Design Review (VADR) Sample Report. Retrieved from https:\/\/www.cisa.gov\/resources-tools\/resources\/validated-architecture-design-review-vadr-sample-report"},{"key":"e_1_3_1_12_2","unstructured":"Cybersecurity and Infrastructure Security Agency (CISA). 2026. Validated Architecture Design Review (VADR) Training. Retrieved from https:\/\/www.cisa.gov\/resources-tools\/training\/validated-architecture-design-review-vadr-training"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.2172\/1341416"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.5555\/AAI29424473"},{"key":"e_1_3_1_15_2","unstructured":"Richard Derbyshire. 2025. Industrial Ouroboros: Deep Lateral Movement via Living Off the Plant. arXiv:2512.21248. Retrieved from https:\/\/arxiv.org\/abs\/2512.21248"},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1109\/EuroSPW.2018.00028","volume-title":"Proceedings of the 2018 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)","author":"Derbyshire Richard","year":"2018","unstructured":"Richard Derbyshire, Benjamin Green, Daniel Prince, Andreas Mauthe, and David Hutchison. 2018. An analysis of cyber security attack taxonomies. In Proceedings of the 2018 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW). IEEE, 153\u2013161."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3670695"},{"key":"e_1_3_1_18_2","volume-title":"Survey Research by Telephone","author":"Frey James H.","year":"1983","unstructured":"James H. Frey. 1983. Survey Research by Telephone. SAGE Publications."},{"issue":"3","key":"e_1_3_1_19_2","first-page":"9","article-title":"Cultural \u201cinsiders\u201d and the issue of positionality in qualitative migration research: Moving \u201cacross\u201d and moving \u201calong\u201d researcher\u2013participant divides","volume":"7","author":"Ganga Deianira","year":"2006","unstructured":"Deianira Ganga and Sam Scott. 2006. Cultural \u201cinsiders\u201d and the issue of positionality in qualitative migration research: Moving \u201cacross\u201d and moving \u201calong\u201d researcher\u2013participant divides. Forum Qualitative Sozialforschung \/ Forum: Qualitative Social Research 7, 3, Article 7 (2006), 9. Retrieved September 10, 2025 from https:\/\/www.qualitative-research.net\/index.php\/fqs\/article\/view\/134","journal-title":"Forum Qualitative Sozialforschung \/ Forum: Qualitative Social Research"},{"key":"e_1_3_1_20_2","volume-title":"The Discovery of Grounded Theory: Strategies for Qualitative Research","author":"Glaser Barney G.","year":"1967","unstructured":"Barney G. Glaser and Anselm L. Strauss. 1967. The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine."},{"key":"e_1_3_1_21_2","volume-title":"Proceedings of the 13th USENIX Workshop on Cyber Security Experimentation and Test (CSET \u201920)","author":"Green Benjamin","year":"2020","unstructured":"Benjamin Green, Ric Derbyshire, William Knowles, James Boorman, Pierre Ciholas, Daniel Prince, and David Hutchison. 2020. ICS testbed Tetris: Practical building blocks towards a cyber security resource. In Proceedings of the 13th USENIX Workshop on Cyber Security Experimentation and Test (CSET \u201920)."},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2021.102424"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3140241.3140254"},{"key":"e_1_3_1_24_2","unstructured":"Eric M. Hutchins Michael J. Cloppert and Rohan M. Amin. 2011. Intelligence-Driven Computer Network Defense Informed by Analysis of Adversary Campaigns and Intrusion Kill Chains. Technical Report. Lockheed Martin 1\u201312. Retrieved from https:\/\/www.lockheedmartin.com\/content\/dam\/lockheed-martin\/rms\/documents\/cyber\/LM-White-Paper-Intel-Driven-Defense.pdfWhitepaper"},{"key":"e_1_3_1_25_2","unstructured":"International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC). 2024. ISO\/IEC 27019:2024: Information Security Cybersecurity and Privacy Protection\u2014Information Security Controls for the Energy Utility Industry."},{"key":"e_1_3_1_26_2","unstructured":"International Society of Automation (ISA) and International Electrotechnical Commission (IEC). 2018. ISA\/IEC 62443-4-1: Security for Industrial Automation and Control Systems\u2014Part 4\u20131: Secure Product Development Lifecycle Requirements."},{"key":"e_1_3_1_27_2","unstructured":"International Society of Automation (ISA) and International Electrotechnical Commission (IEC). 2022. ISA\/IEC 62443-2-1: Security for Industrial Automation and Control Systems\u2014Part 2\u20131: Establishing an IACS Security Program."},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","unstructured":"Karen Scarfone Murat Souppaya Amanda Cody and Angela Orebaugh. 2008. Technical Guide to Information Security Testing and Assessment. Technical Report NIST SP 800-115. National Institute of Standards and Technology. DOI: 10.6028\/NIST.SP.800-115","DOI":"10.6028\/NIST.SP.800-115"},{"key":"e_1_3_1_29_2","first-page":"118","volume-title":"Qualitative Methods and Analysis in Organizational Research: A Practical Guide","author":"King Nigel","year":"1998","unstructured":"Nigel King. 1998. Template Analysis. In Qualitative Methods and Analysis in Organizational Research: A Practical Guide, Gillian Symon and Catherine Cassell (Eds.), SAGE Publications, 118\u2013134."},{"key":"e_1_3_1_30_2","volume-title":"Content Analysis: An Introduction to Its Methodology","author":"Krippendorff Klaus","year":"2004","unstructured":"Klaus Krippendorff. 2004. Content Analysis: An Introduction to Its Methodology (2nd ed.). Sage Publications.","edition":"2"},{"key":"e_1_3_1_31_2","unstructured":"Marina Krotofil. 2024. Industrial Control Systems: Engineering Foundations and the Cyber-Physical Attack Lifecycle. Retrieved September 10 2025 from https:\/\/www.cyberphysicalsecurity.info\/Whitepaper"},{"key":"e_1_3_1_32_2","volume-title":"The Long Interview. Qualitative Research Methods","author":"Ken David McCrac","year":"1988","unstructured":"David McCrac Ken. 1988. The Long Interview. Qualitative Research Methods, Vol. 13. SAGE Publications."},{"key":"e_1_3_1_33_2","unstructured":"MITRE Corporation. 2024. ATT&CK for ICS: Adversarial Tactics Techniques and Common Knowledge for Industrial Control Systems. Retrieved September 10 2025 from https:\/\/attack.mitre.org\/matrices\/ics\/"},{"key":"e_1_3_1_34_2","unstructured":"MITRE Corporation. 2025. Common Vulnerabilities and Exposures (CVE). Retrieved September 10 2025 from https:\/\/www.cve.org"},{"key":"e_1_3_1_35_2","unstructured":"Konstantinos Moulinos and Adrian Pauna. 2013. Good Practices for an EU ICS Testing Coordination Capability. Retrieved September 10 2025 from https:\/\/www.enisa.europa.eu\/sites\/default\/files\/publications\/Good%20practices%20for%20an%20EU%20ICS%20testing%20coordination%20capability.pdf"},{"key":"e_1_3_1_36_2","unstructured":"National Cyber Security Centre. 2022. Cyber-Physical Problem Book: Confidence in the Security of Systems. Retrieved September 9 2025 from https:\/\/www.ncsc.gov.uk\/collection\/problem-book\/cyber-physical\/cp4-confidence-security-systems"},{"key":"e_1_3_1_37_2","unstructured":"National Institute of Standards and Technology (NIST). 2023. Guide to Operational Technology (OT) Security\u2014NIST SP 800\u201382 Revision 3. Technical Report SP 800\u201382 Rev. 3. NIST. Retrieved September 10 2025 from https:\/\/nvlpubs.nist.gov\/nistpubs\/SpecialPublications\/NIST.SP.800-82r3.pdf"},{"key":"e_1_3_1_38_2","unstructured":"North American Electric Reliability Corporation (NERC). 2015. CIP-010-1: Cyber Security\u2014Configuration Change Management and Vulnerability Assessments. Penetration testing discretionary per related FAQs. Retrieved from https:\/\/www.regie-energie.qc.ca\/storage\/app\/media\/entites-visees-normes-de-fiabilite\/english\/reliability-standards\/CIP-010-1-en-2017-04-04.pdf"},{"key":"e_1_3_1_39_2","volume-title":"Interviewing in Educational Research","author":"Powney Janet","year":"1987","unstructured":"Janet Powney and Mike Watts. 1987. Interviewing in Educational Research. Routledge & Kegan Paul, London."},{"key":"e_1_3_1_40_2","volume-title":"Pentesting Industrial Control Systems: An Ethical Hacker\u2019s Guide to Analyzing, Compromising, Mitigating, and Securing Industrial Processes","author":"Smith","year":"2021","unstructured":"Paul Smith. 2021. Pentesting Industrial Control Systems: An Ethical Hacker\u2019s Guide to Analyzing, Compromising, Mitigating, and Securing Industrial Processes. Packt Publishing."},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3569958"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ssci.2024.106481"},{"key":"e_1_3_1_43_2","first-page":"1","volume-title":"Proceedings of the 2023 IFIP Networking Conference (IFIP Networking)","author":"Staves Alexander","year":"2023","unstructured":"Alexander Staves, Sam Maesschalck, Richard Derbyshire, Benjamin Green, and David Hutchison. 2023. Learning to walk: Towards assessing the maturity of OT security control standards and guidelines. In Proceedings of the 2023 IFIP Networking Conference (IFIP Networking). IEEE, 1\u20136."},{"key":"e_1_3_1_44_2","unstructured":"US Department of Homeland Security and Centre for the Protection of National Infrastructure. 2011. Cyber Security Assessments of Industrial Control Systems: A Good Practice Guide. Guidance Document. Retrieved September 9 2025 from https:\/\/www.ccn-cert.cni.es\/publico\/InfraestructurasCriticaspublico\/CPNI-Guia-SCI.pdf"}],"container-title":["Digital Threats: Research and Practice"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3802592","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T14:17:03Z","timestamp":1775571423000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3802592"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,7]]},"references-count":43,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3802592"],"URL":"https:\/\/doi.org\/10.1145\/3802592","relation":{},"ISSN":["2692-1626","2576-5337"],"issn-type":[{"value":"2692-1626","type":"print"},{"value":"2576-5337","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,7]]},"assertion":[{"value":"2025-10-08","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-02-26","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}