{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T15:28:16Z","timestamp":1774625296639,"version":"3.50.1"},"reference-count":56,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,1,27]],"date-time":"2023-01-27T00:00:00Z","timestamp":1674777600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100005416","name":"Norges Forskningsr\u00e5d","doi-asserted-by":"publisher","award":["302941"],"award-info":[{"award-number":["302941"]}],"id":[{"id":"10.13039\/501100005416","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006754","name":"Army Research Laboratory","doi-asserted-by":"publisher","award":["W911NF-17-2-0083"],"award-info":[{"award-number":["W911NF-17-2-0083"]}],"id":[{"id":"10.13039\/100006754","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000104","name":"National Aeronautics and Space Administration","doi-asserted-by":"publisher","award":["80GSFC21M0002"],"award-info":[{"award-number":["80GSFC21M0002"]}],"id":[{"id":"10.13039\/100000104","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Big Data"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Cyber defense decision-making during cyber threat situations is based on human-to-human communication aiming to establish a shared cyber situational awareness. Previous studies suggested that communication inefficiencies were among the biggest problems facing security operation center teams. There is a need for tools that allow for more efficient communication of cyber threat information between individuals both in education and during cyber threat situations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      In the present study, we compared how the visual representation of network topology and traffic in 3D mixed reality vs. 2D affected team performance in a sample of cyber cadets (\n                      <jats:italic>N<\/jats:italic>\n                      = 22) cooperating in dyads. Performance outcomes included network topology recognition, cyber situational awareness, confidence in judgements, experienced communication demands, observed verbal communication, and forced choice decision-making. The study utilized network data from the NATO CCDCOE 2022 Locked Shields cyber defense exercise.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We found that participants using the 3D mixed reality visualization had better cyber situational awareness than participants in the 2D group. The 3D mixed reality group was generally more confident in their judgments except when performing worse than the 2D group on the topology recognition task (which favored the 2D condition). Participants in the 3D mixed reality group experienced less communication demands, and performed more verbal communication aimed at establishing a shared mental model and less communications discussing task resolution. Better communication was associated with better cyber situational awareness. There were no differences in decision-making between the groups. This could be due to cohort effects such as formal training or the modest sample size.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>This is the first study comparing the effect of 3D mixed reality and 2D visualizations of network topology on dyadic cyber team communication and cyber situational awareness. Using 3D mixed reality visualizations resulted in better cyber situational awareness and team communication. The experiment should be repeated in a larger and more diverse sample to determine its potential effect on decision-making.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fdata.2023.1042783","type":"journal-article","created":{"date-parts":[[2023,1,27]],"date-time":"2023-01-27T00:58:24Z","timestamp":1674781104000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["A 3D mixed reality visualization of network topology and activity results in better dyadic cyber team communication and cyber situational awareness"],"prefix":"10.3389","volume":"6","author":[{"given":"Torvald F.","family":"Ask","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kaur","family":"Kullman","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stefan","family":"S\u00fctterlin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benjamin J.","family":"Knox","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Don","family":"Engel","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ricardo G.","family":"Lugo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,1,27]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1080\/23742917.2019.1698178","article-title":"Challenges and performance metrics for security operations center analysts: a systematic review","volume":"4","author":"Agyepong","year":"2019","journal-title":"J. Cyber Security Technol"},{"key":"B2","doi-asserted-by":"crossref","DOI":"10.1109\/CyberSA.2016.7503279","article-title":"\u201cOn the collaborative practices of cyber threat intelligence analysts to develop and utilize tacit threat and defence knowledge,\u201d","volume-title":"2016 International Conference on Cyber Situational Awareness, Data Analytics and Assessment (CyberSA)","author":"Ahrend","year":"2016"},{"key":"B3","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1146\/annurev.neuro.31.060407.125555","article-title":"Vestibular system: the many facets of a multimodal sense","volume":"31","author":"Angelaki","year":"2008","journal-title":"Ann. Rev Neurosci."},{"key":"B4","doi-asserted-by":"publisher","first-page":"1092056","DOI":"10.3389\/fnhum.2022.1092056","article-title":"Neurophysiological and emotional influences on team communication and metacognitive cyber situational awareness during a cyber engineering exercise","volume":"16","author":"Ask","year":"2023","journal-title":"Front. Human Neurosci."},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.31234\/osf.io\/sphgn","article-title":"A 3D mixed reality visualization of network topology and activity results in better dyadic cyber team communication and cyber situational awareness","author":"Ask","year":"2022","journal-title":"PsyArXiv [Preprint]"},{"key":"B6","first-page":"21","article-title":"\u201cHuman-human communication in cyber threat situations: a systematic review,\u201d","volume-title":"HCI International 2021 - Late Breaking Papers: Cognition, Inclusion, Learning and Culture. HCII 2021","author":"Ask","year":""},{"key":"B7","first-page":"3","article-title":"\u201cSituational states influence on team workload demands in cyber defense exercise,\u201d","volume-title":"HCI International 2021 - Late Breaking Papers: Cognition, Inclusion, Learning and Culture. HCII 2021","author":"Ask","year":""},{"key":"B8","first-page":"3","article-title":"\u201cCyber SA: situational awareness for cyber defense,\u201d","volume-title":"Cyber Situational Awareness Advances in Information Security","author":"Barford","year":"2009"},{"key":"B9","doi-asserted-by":"publisher","first-page":"e0438","DOI":"10.1523\/ENEURO.0438-17.2018","article-title":"Development of parvalbumin- expressing basket terminals in layer II of the rat medial entorhinal cortex","volume":"5","author":"Berggaard","year":"2018","journal-title":"eNeuro"},{"key":"B10","doi-asserted-by":"publisher","first-page":"14415","DOI":"10.1038\/ncomms14415","article-title":"Low-frequency theta oscillations in the human hippocampus during real-world and virtual navigation","volume":"8","author":"Bohbot","year":"2017","journal-title":"Nat. Commun"},{"key":"B11","doi-asserted-by":"publisher","first-page":"937","DOI":"10.3389\/fpsyg.2016.00937","article-title":"Mission command in the age of network-enabled operations: social network analysis of information sharing and situation awareness. Front","volume":"7","author":"Buchler","year":"2016","journal-title":"Psychol"},{"key":"B12","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1109\/CogSIMA.2012.6188386","article-title":"\u201cTeam-based cyber defense analysis,\u201d","volume-title":"2012 IEEE International Multi-Disciplinary Conference on Cognitive Methods in situation Awareness and Decision Support","author":"Champion","year":"2012"},{"key":"B13","doi-asserted-by":"publisher","first-page":"284","DOI":"10.1037\/1040-3590.6.4.284","article-title":"Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology","volume":"6","author":"Cicchetti","year":"1994","journal-title":"Psychol. Assess."},{"key":"B14","doi-asserted-by":"publisher","first-page":"732","DOI":"10.1038\/nrn3827","article-title":"Time cells in the hippocampus: a new dimension for mapping memories","volume":"15","author":"Eichenbaum","year":"2014","journal-title":"Nat. Rev. Neurosci."},{"key":"B15","first-page":"97","article-title":"Design and evaluation for situation awareness enhancement","volume-title":"Proceedings of the Human Factors Society 32nd Annual Meeting","author":"Endsley","year":"1988"},{"key":"B16","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1518\/001872095779049543","article-title":"Toward a theory of Situation Awareness in dynamic systems. J. Hum. Factors Ergon","volume":"37","author":"Endsley","year":"1995","journal-title":"Soc"},{"key":"B17","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1177\/1555343419874248","article-title":"The divergence of objective and subjective situation awareness: a meta- analysis","volume":"14","author":"Endsley","year":"2020","journal-title":"J. Cogn. Eng. Decis. Mak"},{"key":"B18","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1016\/j.cose.2014.06.008","article-title":"Cyber situational awareness - a systematic review of the literature","volume":"46","author":"Franke","year":"2014","journal-title":"Comput. Security"},{"key":"B19","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1177\/1555343412451749","article-title":"The role of working memory in levels of situation awareness","volume":"7","author":"Gutzwiller","year":"2013","journal-title":"J. Cogn. Eng. Decis. Mak."},{"key":"B20","doi-asserted-by":"publisher","first-page":"23","DOI":"10.20982\/tqmp.08.1.p023","article-title":"Computing inter-rater reliability for observational data: An overview and tutorial","volume":"8","author":"Hallgren","year":"2012","journal-title":"Tutor. Quant Methods Psychol."},{"key":"B21","doi-asserted-by":"publisher","first-page":"458","DOI":"10.1177\/1071181312561044","article-title":"Influence of Team Communication and Coordination on the Performance of Teams at the iCTF Competition","volume":"56","author":"Jariwala","year":"2012","journal-title":"Proc. Human Factors Ergon. Soc. Ann. Meet."},{"key":"B22","year":"2021","journal-title":"JASP (Version 0.15)"},{"key":"B23","first-page":"178","article-title":"\u201cExploring the hybrid space,\u201d","volume-title":"Augmented Cognition 2016. Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence)","author":"J\u00f8sok","year":"2016"},{"key":"B24","first-page":"486","article-title":"Macrocognition applied to the Hybrid Space: Team environment, functions and processes in Cyber Operations","volume-title":"International Conference on Augmented Cognition","author":"J\u00f8sok","year":"2017"},{"key":"B25","doi-asserted-by":"publisher","first-page":"875","DOI":"10.3389\/fpsyg.2019.00875","article-title":"Self-regulation and cognitive agility in cyber operations. Front","volume":"10","author":"J\u00f8sok","year":"2019","journal-title":"Psychol"},{"key":"B26","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1080\/08995605.2018.1478546","article-title":"Socio-technical communication: The hybrid space and the OLB model for science-based cyber education","volume":"30","author":"Knox","year":"2018","journal-title":"Milit. Psychol."},{"key":"B27","doi-asserted-by":"crossref","first-page":"330","DOI":"10.1007\/978-3-319-58750-9_46","article-title":"\u201cTowards a cognitive agility index: the role of metacognition in human computer interaction,\u201d","volume-title":"HCI International 2017 - Posters' Extended Abstracts","author":"Knox","year":"2017"},{"key":"B28","doi-asserted-by":"publisher","first-page":"5612","DOI":"10.1038\/s41598-021-84599-7","article-title":"An immersive first-person navigation task for abstract knowledge acquisition","volume":"11","author":"Kuhrt","year":"2021","journal-title":"Sci. Rep."},{"key":"B29","first-page":"257","article-title":"\u201cOperator impressions of 3D visualizations for cybersecurity analysts,\u201d","volume-title":"Proceedings of the 18th European Conference on Cyber Warfare and Security, ECCWS 2019: University of Coimbra, Portugal","author":"Kullman","year":""},{"key":"B30","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-50309-3_30","article-title":"\u201cMental model mapping method for cybersecurity,\u201d","volume-title":"HCI for Cybersecurity, Privacy and Trust. HCII 2020","author":"Kullman","year":"2020"},{"key":"B31","first-page":"369","article-title":"\u201cEnhancing cyber defense situational awareness using 3D visualizations,\u201d","volume-title":"Proceedings of the 13th International Conference on Cyber Warfare and Security ICCWS 2018: National Defense University, Washington DC, USA","author":"Kullman","year":"2018"},{"key":"B32","first-page":"13310","article-title":"\u201cUser interactions in virtual data explorer,\u201d","volume-title":"Augmented Cognition. HCII 2022","author":"Kullman","year":""},{"key":"B33","doi-asserted-by":"publisher","first-page":"37","DOI":"10.46713\/jdst.004.03","article-title":"Interactive stereoscopically perceivable multidimensional data visualizations for cybersecurity","volume":"4","author":"Kullman","year":"","journal-title":"J. Defence Secur. Technol"},{"key":"B34","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1016\/j.ifacol.2019.12.093","article-title":"VR\/MR supporting the future of defensive cyber operations","volume":"52","author":"Kullman","year":"","journal-title":"IFAC-PapersOnLine"},{"key":"B35","unstructured":"LanktonP.\n          Endsley's model of situational awareness [jpg]2007"},{"key":"B36","doi-asserted-by":"publisher","first-page":"380","DOI":"10.1016\/j.actpsy.2012.09.002","article-title":"Building a memory palace in minutes: equivalent memory performance using virtual versus conventional environments with the Method of Loci","volume":"141","author":"Legge","year":"2012","journal-title":"Acta Psychol."},{"key":"B37","doi-asserted-by":"crossref","DOI":"10.1109\/CyberSA.2017.8073388","article-title":"\u201cDevelopment and validation of technique to measure cyber situation awareness,\u201d","volume-title":"2017 International Conference on Cyber Situational Awareness, Data Aanalytics and Assessment (Cyber SA)","author":"Lif","year":"2017"},{"key":"B38","first-page":"223","article-title":"\u201cTeam workload demands influence on cyber detection performance,\u201d","volume-title":"13th International Conference on Naturalistic Decision Making","author":"Lugo","year":"2017"},{"key":"B39","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1007\/978-3-319-91122-9_16","article-title":"Cyber officer profiles and performance factors","volume-title":"International Conference on Engineering Psychology and Cognitive Ergonomics","author":"Lugo","year":"2018"},{"key":"B40","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1515\/jms-2016-0005","article-title":"The moderating influence of self-efficacy on interoceptive ability and counterintuitive decision making in officer cadets","volume":"7","author":"Lugo","year":"2016","journal-title":"J. Mil. Stud"},{"key":"B41","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1146\/annurev.neuro.31.061307.090723","article-title":"Place cells, grid cells, and the brain's spatial representation system","volume":"31","author":"Moser","year":"2008","journal-title":"Ann. Rev Neurosci."},{"key":"B42","unstructured":"NATO Recognizes cyberspace as a \u2018domain of operations\u2019 at the Warsaw summit2016"},{"key":"B43","doi-asserted-by":"publisher","first-page":"e13343","DOI":"10.7554\/eLife.13343.019","article-title":"Structural development and dorsoventral maturation of the medial entorhinal cortex","volume":"5","author":"Ray","year":"2016","journal-title":"eLife"},{"key":"B44","doi-asserted-by":"publisher","first-page":"1065","DOI":"10.1038\/s41593-021-00871-z","article-title":"Enhanced hippocampal theta rhythmicity and emergence of eta oscillation in virtual reality","volume":"24","author":"Safaryan","year":"2021","journal-title":"Nat. Neurosci."},{"key":"B45","doi-asserted-by":"publisher","first-page":"597","DOI":"10.1177\/0018720818780472","article-title":"Hacking the human: The prevalence paradox in cybersecurity","volume":"60","author":"Sawyer","year":"2018","journal-title":"Human Factors."},{"key":"B46","doi-asserted-by":"publisher","first-page":"1616","DOI":"10.1073\/pnas.032662099","article-title":"Oscillatory brain states and learning: Impact of hippocampal theta-contingent training","volume":"99","author":"Seager","year":"2002","journal-title":"Proc. Nat. Acad. Sci USA."},{"key":"B47","doi-asserted-by":"publisher","first-page":"989","DOI":"10.1177\/1541931214581207","article-title":"Development of the team workload questionnaire (TWLQ)","volume":"58","author":"Sellers","year":"2014","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet"},{"key":"B48","doi-asserted-by":"publisher","first-page":"420","DOI":"10.1037\/0033-2909.86.2.420","article-title":"Intraclass correlations: Uses in assessing rater reliability","volume":"86","author":"Shrout","year":"1979","journal-title":"Psychol. Bull."},{"key":"B49","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1002\/hipo.1112","article-title":"Hippocampal spatial representations require vestibular input","volume":"12","author":"Stackman","year":"2002","journal-title":"Hippocampus"},{"key":"B50","article-title":"\u201cCollaborative data analysis and discovery for cyber security,\u201d","volume-title":"SOUPS 2016: Twelfth Symposium on Usable Privacy and Security","author":"Staheli","year":"2016"},{"key":"B51","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1109\/MSP.2015.71","article-title":"Improving cybersecurity incident response team effectiveness using teams-based research","volume":"13","author":"Steinke","year":"2015","journal-title":"IEEE Secur. Privacy"},{"key":"B52","first-page":"103","article-title":"\u201cThe role of IT background for metacognitive accuracy, confidence and overestimation of deep fake recognition skills,\u201d","volume-title":"Augmented Cognition. HCII 2022.","author":"S\u00fctterlin","year":"2022"},{"key":"B53","doi-asserted-by":"publisher","first-page":"33","DOI":"10.20870\/IJVR.2016.16.1.2880","article-title":"Proposition and validation of a questionnaire to measure the user experience in immersive virtual environments","volume":"16","author":"Tcha-Tokey","year":"2016","journal-title":"Int. J. Virtual Real"},{"key":"B54","doi-asserted-by":"crossref","first-page":"774","DOI":"10.1109\/ASONAM.2018.8508410","article-title":"\u201cInformation Requirements for National Level Cyber Situational Awareness,\u201d","volume-title":"2018 IEEE\/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM)","author":"Varga","year":"2018"},{"key":"B55","doi-asserted-by":"publisher","first-page":"eabc7606","DOI":"10.1126\/sciadv.abc7606","article-title":"Durable memories and efficient neural coding through mnemonic training using the method of loci","volume":"7","author":"Wagner","year":"2021","journal-title":"Sci. Adv."},{"key":"B56","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1126\/science.663646","article-title":"Loss of hippocampal theta rhythm results in spatial memory deficit in the rat","volume":"201","author":"Winson","year":"1978","journal-title":"Science"}],"container-title":["Frontiers in Big Data"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2023.1042783\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,27]],"date-time":"2023-01-27T00:58:48Z","timestamp":1674781128000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2023.1042783\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,27]]},"references-count":56,"alternative-id":["10.3389\/fdata.2023.1042783"],"URL":"https:\/\/doi.org\/10.3389\/fdata.2023.1042783","relation":{"has-preprint":[{"id-type":"doi","id":"10.31234\/osf.io\/jsnu8","asserted-by":"object"},{"id-type":"doi","id":"10.31234\/osf.io\/sphgn","asserted-by":"object"}]},"ISSN":["2624-909X"],"issn-type":[{"value":"2624-909X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,27]]},"article-number":"1042783"}}