{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T01:33:11Z","timestamp":1775266391469,"version":"3.50.1"},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,5,27]],"date-time":"2023-05-27T00:00:00Z","timestamp":1685145600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2023,7,31]]},"abstract":"<jats:p>Managers rarely have deep knowledge of cyber security and yet are expected to make decisions with cyber security implications for software-based systems. We investigate the decision-making conversations of seven teams of senior managers from the same organisation as they complete the Decisions &amp; Disruptions cyber security exercise. We use grounded theory to situate our analysis of their decision-making and help us explore how these complex socio-cognitive interactions occur. We have developed a goal-model (using iStar 2.0) of the teams\u2019 dialogue that illustrates what cyber security goals teams identify and how they operationalise their decisions to reach these goals. We complement this with our model of cyber security reasoning that describes how these teams make their decisions, showing how each team members\u2019 experience, intuition, and understanding affects the team\u2019s overall shared reasoning and decision-making.<\/jats:p>\n          <jats:p>\n            Our findings show how managers with little cyber security expertise are able to use logic and traditional risk management thinking to make cyber security decisions. Despite their lack of cyber security\u2013specific training, they demonstrate reasoning that closely resembles the decision-making approaches espoused in cyber security\u2013specific standards (e.g., NIST\/ISO). Our work demonstrates how organisations and practitioners can enrich goal modelling to capture not only\n            <jats:italic>what<\/jats:italic>\n            security goals an organisation has (and\n            <jats:italic>how<\/jats:italic>\n            they can operationalise them) but also\n            <jats:italic>how<\/jats:italic>\n            and\n            <jats:italic>why<\/jats:italic>\n            these goals have been identified. Ultimately, non\u2013cyber security experts can develop their cyber security model based on their current context (and update it when new requirements appear or new incidents happen), whilst capturing their reasoning at every stage.\n          <\/jats:p>","DOI":"10.1145\/3548682","type":"journal-article","created":{"date-parts":[[2022,8,1]],"date-time":"2022-08-01T11:46:47Z","timestamp":1659354407000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Making Sense of the Unknown: How Managers Make Cyber Security Decisions"],"prefix":"10.1145","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9329-4866","authenticated-orcid":false,"given":"Benjamin","family":"Shreeve","sequence":"first","affiliation":[{"name":"University of Bristol, United Kingdom"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3622-7851","authenticated-orcid":false,"given":"Catarina","family":"Gralha","sequence":"additional","affiliation":[{"name":"NOVA LINCS, Universidade NOVA de Lisboa, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0109-1341","authenticated-orcid":false,"given":"Awais","family":"Rashid","sequence":"additional","affiliation":[{"name":"University of Bristol, United Kingdom"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5914-1631","authenticated-orcid":false,"given":"Jo\u00e3o","family":"Ara\u00fajo","sequence":"additional","affiliation":[{"name":"NOVA LINCS, Universidade NOVA de Lisboa, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5356-5203","authenticated-orcid":false,"given":"Miguel","family":"Goul\u00e3o","sequence":"additional","affiliation":[{"name":"NOVA LINCS, Universidade NOVA de Lisboa, Portugal"}]}],"member":"320","published-online":{"date-parts":[[2023,5,27]]},"reference":[{"key":"e_1_3_5_2_2","unstructured":"CCDCOE. CCDCOE Locked Shields Exercise. 2021. Retrieved from https:\/\/ccdcoe.org\/exercises\/locked-shields\/."},{"key":"e_1_3_5_3_2","first-page":"72","article-title":"Work system theory: Overview of core concepts, extensions, and challenges for the future","author":"Alter Steven","year":"2013","unstructured":"Steven Alter. 2013. Work system theory: Overview of core concepts, extensions, and challenges for the future. J. Assoc. Inf. Syst. (2013), 72.","journal-title":"J. Assoc. Inf. Syst."},{"key":"e_1_3_5_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2016.39"},{"key":"e_1_3_5_5_2","volume-title":"Proceedings of the USENIX Workshop on Advances in Security Education (ASE\u201918)","author":"Bock Kevin","year":"2018","unstructured":"Kevin Bock, George Hughey, and Dave Levin. 2018. King of the hill: A novel cybersecurity competition for teaching penetration testing. In Proceedings of the USENIX Workshop on Advances in Security Education (ASE\u201918). USENIX Association, 9."},{"key":"e_1_3_5_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jesp.2003.11.003"},{"key":"e_1_3_5_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/1005817.1005828"},{"key":"e_1_3_5_8_2","unstructured":"Fabiano Dalpiaz Xavier Franch and Jennifer Horkoff. 2016. iStar 2.0 Language Guide. Retrieved from https:\/\/arxiv.org\/abs\/1605.07767v3."},{"key":"e_1_3_5_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/2508859.2516753"},{"key":"e_1_3_5_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2016.09.006"},{"key":"e_1_3_5_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-75563-0_26"},{"key":"e_1_3_5_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3196884"},{"key":"e_1_3_5_13_2","article-title":"The good, the bad and the ugly: A study of security decisions in a cyber-physical systems game","author":"Frey Sylvain","year":"2017","unstructured":"Sylvain Frey, Awais Rashid, Pauline Anthonysamy, Maria Pinto-Albuquerque, and Syed Asad Naqvi. 2017. The good, the bad and the ugly: A study of security decisions in a cyber-physical systems game. IEEE Trans. Softw. Eng. (2017).","journal-title":"IEEE Trans. Softw. Eng."},{"key":"e_1_3_5_14_2","doi-asserted-by":"publisher","DOI":"10.2307\/256117"},{"key":"e_1_3_5_15_2","volume-title":"The Discovery of Grounded Theory: Strategies for Qualitative Research","author":"Glaser Barney G.","year":"1967","unstructured":"Barney G. Glaser and Anselm L. Strauss. 1967. The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine de Gruyter, New York."},{"key":"e_1_3_5_16_2","first-page":"34","article-title":"Vital lies","author":"Goleman Daniel","year":"1985","unstructured":"Daniel Goleman. 1985. Vital lies. Simple Truths (1985), 34\u201336.","journal-title":"Simple Truths"},{"key":"e_1_3_5_17_2","volume-title":"Proceedings of the 6th Workshop on Cyber Security Experimentation and Test","author":"Gondree Mark","year":"2013","unstructured":"Mark Gondree and Zachary N. J. Peterson. 2013. Valuing security by getting [d0x3d!]: Experiences with a network security board game. In Proceedings of the 6th Workshop on Cyber Security Experimentation and Test. USENIX, 8."},{"key":"e_1_3_5_18_2","volume-title":"ISO 27035-1:2016: Information Technology\u2014Security Techniques\u2014Information Security Incident Management","year":"2016","unstructured":"ISO. 2016. ISO 27035-1:2016: Information Technology\u2014Security Techniques\u2014Information Security Incident Management. Standard. International Organization for Standardization."},{"key":"e_1_3_5_19_2","volume-title":"ISO\/IEC 27001","year":"2013","unstructured":"ISO\/IEC. 2013. ISO\/IEC 27001. Technical Report."},{"key":"e_1_3_5_20_2","volume-title":"ISO\/IEC FDIS 25023:2016(E) Standard: Systems and Software Engineering\u2014Systems and Software Quality Requirements and Evaluation (SQuaRE)\u2014Measurement of System and Software Product Quality","year":"2016","unstructured":"ISO. 2016. ISO\/IEC FDIS 25023:2016(E) Standard: Systems and Software Engineering\u2014Systems and Software Quality Requirements and Evaluation (SQuaRE)\u2014Measurement of System and Software Product Quality. ISO, Geneva, Switzerland."},{"key":"e_1_3_5_21_2","volume-title":"Identifying How Firms Manage Cybersecurity Investment","author":"Moore Tyler","year":"2015","unstructured":"Tyler Moore, Scott Dynes, and Frederick R. Chang. 2015. Identifying How Firms Manage Cybersecurity Investment. Technical Report. Darwin Deason Institute for Cybersecurity, Southern Methodist University."},{"key":"e_1_3_5_22_2","volume-title":"Proceedings of the USENIX Workshop on Advances in Security Education (ASE\u201918)","author":"Morelock John R.","year":"2018","unstructured":"John R. Morelock and Zachary Peterson. 2018. Authenticity, ethicality, and motivation: A formal evaluation of a 10-week computer security alternate reality game for CS undergraduates. In Proceedings of the USENIX Workshop on Advances in Security Education (ASE\u201918). USENIX Association, 11."},{"key":"e_1_3_5_23_2","doi-asserted-by":"publisher","DOI":"10.5555\/1841349.1841355"},{"key":"e_1_3_5_24_2","unstructured":"National Buraeu of Standards Federal Information Processing Standards Publications (FIPS PUB) 65. 1975. Guideline for Automatic Data Processing Risk Analysis ."},{"key":"e_1_3_5_25_2","first-page":"443","volume-title":"Proceedings of the 14th Symposium on Usable Privacy and Security (SOUPS\u201918)","author":"Nicholson James","year":"2018","unstructured":"James Nicholson, Lynne Coventry, and Pam Briggs. 2018. Introducing the cybersurvival task: assessing and addressing staff beliefs about effective cyber protection. In Proceedings of the 14th Symposium on Usable Privacy and Security (SOUPS\u201918). 443\u2013457."},{"key":"e_1_3_5_26_2","volume-title":"NIST 800-61 Revision 2: Computer Security Incident Handling Guide","year":"2012","unstructured":"NIST. 2012. NIST 800-61 Revision 2: Computer Security Incident Handling Guide. Standard. National Institute of Standards and Technology."},{"key":"e_1_3_5_27_2","first-page":"13","volume-title":"Proceedings of the 10th IASTED International Conference on Software Engineering and Applications (SEA\u201906)","author":"Oladimeji Ebenezer A.","year":"2006","unstructured":"Ebenezer A. Oladimeji, Sam Supakkul, and Lawrence Chung. 2006. Security threat modeling and analysis: A goal-oriented approach. In Proceedings of the 10th IASTED International Conference on Software Engineering and Applications (SEA\u201906). Citeseer, 13\u201315."},{"key":"e_1_3_5_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/HASE.2008.57"},{"key":"e_1_3_5_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2018.00071"},{"key":"e_1_3_5_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-45480-1_27"},{"key":"e_1_3_5_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2011.02.013"},{"key":"e_1_3_5_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/169059.169209"},{"key":"e_1_3_5_33_2","volume-title":"Proceedings of the USENIX Summit on Gaming, Games, and Gamification in Security Education (3GSE\u201914)","author":"Shostack Adam","year":"2014","unstructured":"Adam Shostack. 2014. Elevation of privilege: Drawing developers into threat modeling. In Proceedings of the USENIX Summit on Gaming, Games, and Gamification in Security Education (3GSE\u201914)."},{"key":"e_1_3_5_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3419101"},{"key":"e_1_3_5_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2020.3023735"},{"key":"e_1_3_5_36_2","unstructured":"Benjamin Shreeve and Awais Rashid. Decisions & Disruptions. Retrieved January 2022 from https:\/\/www.decisions-disruptions.org\/."},{"key":"e_1_3_5_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-84800-044-5_12"},{"key":"e_1_3_5_38_2","first-page":"33","volume-title":"The Executive Effect: Concepts and Methods for Studying Top Managers","author":"Starbuck William H.","year":"1988","unstructured":"William H. Starbuck and Frances J. Milliken. 1988. Executives\u2019 perceptual filters: What they notice and how they make sense. In The Executive Effect: Concepts and Methods for Studying Top Managers, Frances J. Milliken and William H. Starbuck (Eds.). 33\u201365."},{"key":"e_1_3_5_39_2","first-page":"621","volume-title":"Proceedings of the 27th USENIX Security Symposium (USENIX Security\u201918)","author":"Stevens Rock","year":"2018","unstructured":"Rock Stevens, Daniel Votipka, Elissa M. Redmiles, Colin Ahern, Patrick Sweeney, and Michelle L. Mazurek. 2018. The battle for New York: A case study of applied digital threat modeling at the enterprise level. In Proceedings of the 27th USENIX Security Symposium (USENIX Security\u201918). USENIX Association, Stanford, CA, 621\u2013637."},{"key":"e_1_3_5_40_2","volume-title":"Basic of Qualitative Research: Techniques and Procedures for Developing Grounded Theory","author":"Strauss Anselm L.","year":"1998","unstructured":"Anselm L. Strauss and Juliet M. Corbin. 1998. Basic of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Sage."},{"key":"e_1_3_5_41_2","volume-title":"Sensemaking in Organizations","author":"Weick Karl E.","year":"1995","unstructured":"Karl E. Weick. 1995. Sensemaking in Organizations. Vol. 3. Sage."},{"key":"e_1_3_5_42_2","volume-title":"Proceedings of the Digital Games Research Association International Conference (DiGRA\u201911)","author":"Xu Yan","year":"2011","unstructured":"Yan Xu, Evan Barba, Iulian Radu, Maribeth Gandy, and Blair MacIntyre. 2011. Chores are fun: Understanding social play in board games for digital tabletop game design. In Proceedings of the Digital Games Research Association International Conference (DiGRA\u201911)."},{"key":"e_1_3_5_43_2","volume-title":"Modelling Strategic Relationships for Process Reengineering","author":"Yu Eric","year":"1995","unstructured":"Eric Yu. 1995. Modelling Strategic Relationships for Process Reengineering. Ph.D. Dissertation. University of Toronto, Canada."},{"key":"e_1_3_5_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/1456362.1456366"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3548682","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3548682","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:50:52Z","timestamp":1750182652000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3548682"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,27]]},"references-count":43,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,7,31]]}},"alternative-id":["10.1145\/3548682"],"URL":"https:\/\/doi.org\/10.1145\/3548682","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,27]]},"assertion":[{"value":"2021-03-16","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-06-20","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-05-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}