{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T08:15:34Z","timestamp":1773735334529,"version":"3.50.1"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T00:00:00Z","timestamp":1773705600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T00:00:00Z","timestamp":1773705600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Centre for Research & Technology Hellas"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int. J. Inf. Secur."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>As cyber-attacks grow in scale, sophistication, and impact, advanced training and evaluation platforms are essential for cybersecurity readiness. Cyber Ranges (CRs) provide controlled, high-fidelity simulation environments in which professionals can practice defensive strategies under realistic threat scenarios. In such environments, success depends not only on technical skill but also on robust Situational Awareness (SA), the cognitive capacity to perceive events, understand their meaning, and project future developments. Although SA is recognised as critical, existing CR evaluation methods typically focus on overall performance metrics (e.g., task completion rates, response accuracy, mission success). Such assessments often fail to capture the deeper, cognitive aspects of SA, limiting the CR\u2019s ability to evaluate decision-making agility and strategic insight. In response, this work presents the SA Scoring System (SASS), a dynamic, multi-dimensional framework designed to quantify SA in live CR environments. SASS decomposes SA into core dimensions, perception, comprehension, projection, and resolution, then monitors trainee behaviour and scenario interactions in real-time. Each dimension is assessed via weighted metrics: timing of responses, sequence and number of reasoning steps, usage and effectiveness of tools and techniques, hints requested, and decision outcomes. These metrics are aggregated into interpretable scores that reflect both aspects, specific performance and overall SA proficiency. In this study, the SASS is validated in a series of cyber-defence exercises involving participants of differing expertise levels. The results demonstrate that SASS scores correlate strongly with expert evaluations and can discriminate between high and low SA performers with high fidelity. SASS enables instructors to pinpoint cognitive weak spots (e.g. delayed perception or faulty projection) and adapt training on the fly. Ultimately, integrating the SASS into CR platforms enriches the evaluation process, promoting not just technical competence but also the situational agility needed to counter modern cyber threats.<\/jats:p>","DOI":"10.1007\/s10207-026-01219-z","type":"journal-article","created":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T06:03:31Z","timestamp":1773727411000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Measuring the unseen: a situational awareness scoring framework for cyber range training"],"prefix":"10.1007","volume":"25","author":[{"given":"Amalia","family":"Damianou","sequence":"first","affiliation":[]},{"given":"Mariana S.","family":"Mazi","sequence":"additional","affiliation":[]},{"given":"George","family":"Rizos","sequence":"additional","affiliation":[]},{"given":"Konstantinos","family":"Bezas","sequence":"additional","affiliation":[]},{"given":"George","family":"Lazaridis","sequence":"additional","affiliation":[]},{"given":"Antonis","family":"Voulgaridis","sequence":"additional","affiliation":[]},{"given":"Konstantinos","family":"Votis","sequence":"additional","affiliation":[]},{"given":"Dimitrios","family":"Tzovaras","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2026,3,17]]},"reference":[{"key":"1219_CR1","doi-asserted-by":"crossref","unstructured":"Damianou, A., Mazi, M.S., Rizos, G., Voulgaridis, A., Votis, K.: Situational awareness scoring system in cyber range platforms. In: 2024 IEEE International Conference on Cyber Security and Resilience (CSR), 520\u2013525 (2024). IEEE","DOI":"10.1109\/CSR61664.2024.10679451"},{"issue":"2","key":"1219_CR2","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1207\/s15327108ijap0302_5","volume":"3","author":"MR Endsley","year":"1993","unstructured":"Endsley, M.R.: A survey of situation awareness requirements in air-to-air combat fighters. Int. J. Aviat. Psychol. 3(2), 157\u2013168 (1993)","journal-title":"Int. J. Aviat. Psychol."},{"key":"1219_CR3","unstructured":"ECSO: Understanding Cyber Ranges: From Hype to Reality. https:\/\/ecs-org.eu\/?publications=https-ecs-org-eu-documents-publications-5fdb291cdf5e7-pdf"},{"key":"1219_CR4","unstructured":"NIST: The Cyber Range: A Guide. Accessed: Oct 20, 2025"},{"key":"1219_CR5","doi-asserted-by":"crossref","unstructured":"Aaltola, K.: Empirical study on cyber range capabilities, interactions and learning features. Digital Transformation, Cyber Security and Resilience of Modern Societies, 413\u2013428 (2021)","DOI":"10.1007\/978-3-030-65722-2_26"},{"key":"1219_CR6","doi-asserted-by":"crossref","unstructured":"Mengidis, N., Bozhilova, M., Ceresola, C., Colabuono, C., Cooke, M., Depaix, G., Genchev, A., Koykov, G., Mees, W., Merialdo, M., et al.: Leveraging cyber ranges for prototyping, certification and training: The echo case. In: 2022 IEEE International Conference on Cyber Security and Resilience (CSR), 299\u2013304 (2022). IEEE","DOI":"10.1109\/CSR54599.2022.9850278"},{"key":"1219_CR7","unstructured":"Graziano, A.: About federation of cyber ranges, market places and technology innovation. Jun (2020)"},{"key":"1219_CR8","doi-asserted-by":"crossref","unstructured":"Oikonomou, N., Mengidis, N., Spanopoulos-Karalexidis, M., Voulgaridis, A., Merialdo, M., Raisr, I., Hanson, K., La\u00a0Vallee, P., Tsikrika, T., Vrochidis, S., et al.: Echo federated cyber range: towards next-generation scalable cyber ranges. In: 2021 IEEE International Conference on Cyber Security and Resilience (CSR), 403\u2013408 (2021). IEEE","DOI":"10.1109\/CSR51186.2021.9527985"},{"key":"1219_CR9","doi-asserted-by":"crossref","unstructured":"Diakoumakos, J., Chaskos, E., Kolokotronis, N., Lepouras, G.: Cyber-range federation and cyber-security games: a gamification scoring model. In: 2021 IEEE International Conference on Cyber Security and Resilience (CSR), 186\u2013191 (2021). IEEE","DOI":"10.1109\/CSR51186.2021.9527972"},{"key":"1219_CR10","unstructured":"Davis, J., Magrath, S.: A survey of cyber ranges and testbeds. (2013)"},{"issue":"2014","key":"1219_CR11","first-page":"1","volume":"1","author":"M Grieves","year":"2014","unstructured":"Grieves, M.: Digital twin: manufacturing excellence through virtual factory replication. White paper 1(2014), 1\u20137 (2014)","journal-title":"White paper"},{"issue":"11","key":"1219_CR12","doi-asserted-by":"publisher","first-page":"1016","DOI":"10.1016\/j.ifacol.2018.08.474","volume":"51","author":"W Kritzinger","year":"2018","unstructured":"Kritzinger, W., Karner, M., Traar, G., Henjes, J., Sihn, W.: Digital twin in manufacturing: A categorical literature review and classification. Ifac-PapersOnline 51(11), 1016\u20131022 (2018)","journal-title":"Ifac-PapersOnline"},{"key":"1219_CR13","doi-asserted-by":"crossref","unstructured":"Qi, Q., Tao, F.: Digital twin and big data towards smart manufacturing and industry 4.0: 360 degree comparison. Ieee Access 6, 3585\u20133593 (2018)","DOI":"10.1109\/ACCESS.2018.2793265"},{"key":"1219_CR14","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1016\/j.apmt.2018.11.003","volume":"14","author":"T Mukherjee","year":"2019","unstructured":"Mukherjee, T., DebRoy, T.: A digital twin for rapid qualification of 3d printed metallic components. Appl. Mater. Today 14, 59\u201365 (2019)","journal-title":"Appl. Mater. Today"},{"key":"1219_CR15","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1016\/j.cose.2014.06.008","volume":"46","author":"U Franke","year":"2014","unstructured":"Franke, U., Brynielsson, J.: Cyber situational awareness-a systematic review of the literature. Computers & security 46, 18\u201331 (2014)","journal-title":"Computers & security"},{"issue":"2","key":"1219_CR16","doi-asserted-by":"publisher","first-page":"822","DOI":"10.3390\/s23020822","volume":"23","author":"IB Mirza","year":"2023","unstructured":"Mirza, I.B., Georgakopoulos, D., Yavari, A.: Cyber-physical-social awareness platform for comprehensive situation awareness. Sensors 23(2), 822 (2023)","journal-title":"Sensors"},{"issue":"1","key":"1219_CR17","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1518\/001872095779049543","volume":"37","author":"MR Endsley","year":"1995","unstructured":"Endsley, M.R.: Toward a theory of situation awareness in dynamic systems. Hum. Factors 37(1), 32\u201364 (1995)","journal-title":"Hum. Factors"},{"key":"1219_CR18","doi-asserted-by":"publisher","unstructured":"Jajodia, S., Liu, P., Swarup, V., Wang, C. (eds.): Cyber Situational Awareness. Advances in Information Security, 46. Springer, Boston, MA (2010). https:\/\/doi.org\/10.1007\/978-1-4419-0140-8","DOI":"10.1007\/978-1-4419-0140-8"},{"issue":"1","key":"1219_CR19","doi-asserted-by":"publisher","first-page":"640","DOI":"10.1109\/COMST.2018.2871866","volume":"21","author":"M Hus\u00e1k","year":"2018","unstructured":"Hus\u00e1k, M., Kom\u00e1rkov\u00e1, J., Bou-Harb, E., \u010celeda, P.: Survey of attack projection, prediction, and forecasting in cyber security. IEEE Communications Surveys & Tutorials 21(1), 640\u2013660 (2018)","journal-title":"IEEE Communications Surveys & Tutorials"},{"key":"1219_CR20","doi-asserted-by":"crossref","unstructured":"Tianfield, H.: Cyber security situational awareness. In: 2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), 782\u2013787 (2016). IEEE","DOI":"10.1109\/iThings-GreenCom-CPSCom-SmartData.2016.165"},{"key":"1219_CR21","unstructured":"McGuinness, B., Foy, L.: A subjective measure of sa the crew awareness rating scale-getinfo. In: Proceedings of the First Human Performance, Situation Awareness, and Automation Conference., Savannah, Georgia, USA (2000)"},{"key":"1219_CR22","doi-asserted-by":"crossref","unstructured":"Staheli, D., Yu, T., Crouser, R.J., Damodaran, S., Nam, K., O\u2019Gwynn, D., McKenna, S., Harrison, L.: Visualization evaluation for cyber security: Trends and future directions. In: Proceedings of the Eleventh Workshop on Visualization for Cyber Security, 49\u201356 (2014)","DOI":"10.1145\/2671491.2671492"},{"key":"1219_CR23","doi-asserted-by":"crossref","unstructured":"Fink, G., Best, D., Manz, D., Popovsky, V., Endicott-Popovsky, B.: Gamification for measuring cyber security situational awareness. In: International Conference on Augmented Cognition, 656\u2013665 (2013). Springer","DOI":"10.1007\/978-3-642-39454-6_70"},{"key":"1219_CR24","unstructured":"\u010celeda, P., \u010cegan, J., Vykopal, J., Tovar\u0148\u00e1k, D., et al.: Kypo\u2013a platform for cyber defence exercises. M&S Support to Operational Tasks Including War Gaming, Logistics, Cyber Defence. NATO Science and Technology Organization (2015)"},{"key":"1219_CR25","doi-asserted-by":"publisher","first-page":"236","DOI":"10.1007\/s11036-019-01442-0","volume":"25","author":"M Andreolini","year":"2020","unstructured":"Andreolini, M., Colacino, V.G., Colajanni, M., Marchetti, M.: A framework for the evaluation of trainee performance in cyber range exercises. Mobile Networks and Applications 25, 236\u2013247 (2020)","journal-title":"Mobile Networks and Applications"},{"key":"1219_CR26","doi-asserted-by":"publisher","unstructured":"Kampourakis, V., Gkioulos, V., Katsikas, S.: A step-by-step definition of a reference architecture for cyber ranges. Journal of Information Security and Applications 88, 103917 (2025) https:\/\/doi.org\/10.1016\/j.jisa.2024.103917","DOI":"10.1016\/j.jisa.2024.103917"},{"key":"1219_CR27","unstructured":"Bolstad, C.A., Cuevas, H.M.: Integrating situation awareness assessment into humansystems integration"},{"key":"1219_CR28","unstructured":"Tam, K., Moara-Nkwe, K., Jones, K.D.: Maritime technology and research"},{"key":"1219_CR29","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2024.114308","volume":"186","author":"M Pourbehzadi","year":"2024","unstructured":"Pourbehzadi, M., Javidi, G., Howell, C.J., Kamar, E., Sheybani, E.: Enhanced (cyber) situational awareness: Using interpretable principal component analysis (ipca) to automate vulnerability severity scoring. Decis. Support Syst. 186, 114308 (2024)","journal-title":"Decis. Support Syst."},{"key":"1219_CR30","doi-asserted-by":"crossref","unstructured":"Korkiakoski, M., Antila, A., Annamaa, J., Sheikhi, S., Alavesa, P., Kostakos, P.: Hack the room: Exploring the potential of an augmented reality game for teaching cyber security. In: Proceedings of the Augmented Humans International Conference 2023, 349\u2013353 (2023)","DOI":"10.1145\/3582700.3583955"},{"key":"1219_CR31","doi-asserted-by":"publisher","unstructured":"Castillo, J.M., Parenthoen, M., Louveton, N.: A theoretical model for enhancing cyber crisis training in cyber ranges: bridging cognitive load, situational awareness, and immersion. Cognition, Technology & Work (2025) https:\/\/doi.org\/10.1007\/s10111-025-00844-3","DOI":"10.1007\/s10111-025-00844-3"},{"issue":"5","key":"1219_CR32","doi-asserted-by":"publisher","first-page":"737","DOI":"10.1177\/0018720820969071","volume":"65","author":"T Zhang","year":"2023","unstructured":"Zhang, T., Yang, J., Liang, N., Pitts, B.J., Prakah-Asante, K., Curry, R., Duerstock, B., Wachs, J.P., Yu, D.: Physiological measurements of situation awareness: A systematic review. Hum. Factors 65(5), 737\u2013758 (2023). https:\/\/doi.org\/10.1177\/0018720820969071. (PMID: 33241945)","journal-title":"Hum. Factors"},{"key":"1219_CR33","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1201\/b12461","volume-title":"Situation Awareness: Analysis and Measurement","author":"MR Endsley","year":"2000","unstructured":"Endsley, M.R.: Direct measurement of situation awareness: Validity and use of sagat. In: Endsley, M.R., Garland, D.J. (eds.) Situation Awareness: Analysis and Measurement, pp. 147\u2013173. Routledge, Boca Raton, FL (2000)"},{"key":"1219_CR34","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1016\/j.ecns.2020.02.003","volume":"43","author":"D Dishman","year":"2020","unstructured":"Dishman, D., Fallacaro, M.D., Damico, N., Wright, M.C.: Adaptation and validation of the situation awareness global assessment technique for nurse anesthesia graduate students. Clin. Simul. Nurs. 43, 35\u201343 (2020)","journal-title":"Clin. Simul. Nurs."},{"issue":"3","key":"1219_CR35","doi-asserted-by":"publisher","first-page":"388","DOI":"10.1177\/0193945917697230","volume":"40","author":"SB Orique","year":"2018","unstructured":"Orique, S.B., Despins, L.: Evaluating situation awareness: an integrative review. West. J. Nurs. Res. 40(3), 388\u2013424 (2018)","journal-title":"West. J. Nurs. Res."},{"key":"1219_CR36","doi-asserted-by":"crossref","unstructured":"Endsley, M.R., Selcon, S.J., Hardiman, T.D., Croft, D.G.: A comparative analysis of sagat and sart for evaluations of situation awareness. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 42, 82\u201386 (1998). Sage Publications Sage CA: Los Angeles, CA","DOI":"10.1177\/154193129804200119"},{"key":"1219_CR37","unstructured":"Selcon, S.J., Taylor, R.: Evaluation of the situational awareness rating technique(sart) as a tool for aircrew systems design. AGARD, Situational Awareness in Aerospace Operations 8 p(SEE N 90-28972 23-53) (1990)"},{"key":"1219_CR38","doi-asserted-by":"crossref","unstructured":"Selcon, S.J., Taylor, R.M., Koritsas, E.: Workload or situational awareness?: Tlx vs. sart for aerospace systems design evaluation. In: Proceedings of the Human Factors Society Annual Meeting, 35, 62\u201366 (1991). SAGE Publications Sage CA: Los Angeles, CA","DOI":"10.1518\/107118191786755706"},{"key":"1219_CR39","first-page":"259","volume-title":"Macrocognition in Teams: Theories and Methodologies","author":"PW Foltz","year":"2008","unstructured":"Foltz, P.W., Bolstad, C.A., Cuevas, H.M., Franzke, M., Rosenstein, M., Costello, A.M.: Measuring situation awareness through automated communication analysis. In: Letsky, M.P., Warner, N.W., Fiore, S.M., Smith, C.A.P. (eds.) Macrocognition in Teams: Theories and Methodologies, pp. 259\u2013275. CRC Press, Boca Raton, FL (2008)"}],"container-title":["International Journal of Information Security"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10207-026-01219-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10207-026-01219-z","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10207-026-01219-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T06:03:39Z","timestamp":1773727419000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10207-026-01219-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,17]]},"references-count":39,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,4]]}},"alternative-id":["1219"],"URL":"https:\/\/doi.org\/10.1007\/s10207-026-01219-z","relation":{},"ISSN":["1615-5270"],"issn-type":[{"value":"1615-5270","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,17]]},"assertion":[{"value":"31 October 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 January 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 March 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing financial or non-financial interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"73"}}