{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T14:09:51Z","timestamp":1781878191974,"version":"3.54.5"},"reference-count":36,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2020,9,2]],"date-time":"2020-09-02T00:00:00Z","timestamp":1599004800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Digital Threats"],"published-print":{"date-parts":[[2020,9,30]]},"abstract":"<jats:p>Demand is present among security practitioners for improving cyber situational awareness (SA), but capability and assessment have not risen to match. SA is an integral component of cybersecurity for everyone from individuals to business to response teams and threat exchanges. In this Field Note, we highlight existing research and our field observations, a recent review of cyber SA research literature, and call upon the research community to help address three research problems in situational awareness for cybersecurity. The gaps suggest the need to (1) understand what cyber SA is from the human operators\u2019 perspectives, then (2) measure it so that (3) the community can learn whether SA makes a difference in meaningful ways to cybersecurity, and whether methods, technology, or other solutions would improve SA and thus, improve those outcomes.<\/jats:p>","DOI":"10.1145\/3384471","type":"journal-article","created":{"date-parts":[[2020,7,7]],"date-time":"2020-07-07T12:32:18Z","timestamp":1594125138000},"page":"1-6","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":38,"title":["Gaps and Opportunities in Situational Awareness for Cybersecurity"],"prefix":"10.1145","volume":"1","author":[{"given":"Robert","family":"Gutzwiller","sequence":"first","affiliation":[{"name":"Arizona State University, Ira A. Fulton Schools of Engineering (The Polytechnic School), Human Systems Engineering Program, Mesa, AZ, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Josiah","family":"Dykstra","sequence":"additional","affiliation":[{"name":"Cybersecurity Collaboration Center, National Security Agency, Ft. George G. Meade, MD, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bryan","family":"Payne","sequence":"additional","affiliation":[{"name":"Netflix, Los Gatos, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,9,2]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Cyber SA: Situational awareness for cyber defense. In Cyber Situational Awareness","author":"Barford P.","year":"2010","unstructured":"P. Barford , M. Dacier , T. G. Dietterich , M. Fredrikson , J. Giffin , S. Jajodia , and J. Yen . 2010 . Cyber SA: Situational awareness for cyber defense. In Cyber Situational Awareness . Springer , Boston, MA , 3--14. P. Barford, M. Dacier, T. G. Dietterich, M. Fredrikson, J. Giffin, S. Jajodia, and J. Yen. 2010. Cyber SA: Situational awareness for cyber defense. In Cyber Situational Awareness. Springer, Boston, MA, 3--14."},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 310--314","author":"Champion M.","year":"1931","unstructured":"M. Champion , S. Jariwala , P. Ward , and N. J. Cooke . 2014. Using cognitive task analysis to investigate the contribution of informal education to developing cyber security expertise . In Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 310--314 . http:\/\/doi.org\/10.1177\/154 1931 214581064 M. Champion, S. Jariwala, P. Ward, and N. J. Cooke. 2014. Using cognitive task analysis to investigate the contribution of informal education to developing cyber security expertise. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 310--314. http:\/\/doi.org\/10.1177\/1541931214581064"},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the Workshop on Visualization for Computer Security, J. Goodall, G. Conti, and K. Ma (Eds.). Springer","author":"D'Amico A. D.","unstructured":"A. D. D'Amico and K. Whitley . 2008. The real work of computer network defense analysts: The analysis roles and processes that transform network data into security situation awareness . In Proceedings of the Workshop on Visualization for Computer Security, J. Goodall, G. Conti, and K. Ma (Eds.). Springer , Berlin. A. D. D'Amico and K. Whitley. 2008. The real work of computer network defense analysts: The analysis roles and processes that transform network data into security situation awareness. In Proceedings of the Workshop on Visualization for Computer Security, J. Goodall, G. Conti, and K. Ma (Eds.). Springer, Berlin."},{"key":"e_1_2_1_4_1","doi-asserted-by":"crossref","unstructured":"F. T. Durso K. Rawson and S. Girotto. 2007. Comprehension and situation awareness. In Handbook of Applied Cognition F. Durso R. Nickerson S. Dumais S. Lewandowsky and T. Perfect (Eds.). John Wiley & Sons West Sussex 163--193.  F. T. Durso K. Rawson and S. Girotto. 2007. Comprehension and situation awareness. In Handbook of Applied Cognition F. Durso R. Nickerson S. Dumais S. Lewandowsky and T. Perfect (Eds.). John Wiley & Sons West Sussex 163--193.","DOI":"10.1002\/9780470713181.ch7"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the IEEE National Aerospace and Electronics Conference (NAECON\u201988)","author":"Endsley M. R.","year":"1988","unstructured":"M. R. Endsley . 1988 . Situation awareness global assessment technique (SAGAT) . In Proceedings of the IEEE National Aerospace and Electronics Conference (NAECON\u201988) . 789--795. M. R. Endsley. 1988. Situation awareness global assessment technique (SAGAT). In Proceedings of the IEEE National Aerospace and Electronics Conference (NAECON\u201988). 789--795."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1518\/001872095779049499"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10111-015-0323-5"},{"key":"e_1_2_1_8_1","doi-asserted-by":"crossref","unstructured":"M. R. Endsley. 2019. A systematic review and meta-analysis of direct objective measures of situation awareness: A comparison of SAGAT and SPAM. Hum. Fact. https:\/\/doi.org\/10.1177\/0018720819875376  M. R. Endsley. 2019. A systematic review and meta-analysis of direct objective measures of situation awareness: A comparison of SAGAT and SPAM. Hum. Fact. https:\/\/doi.org\/10.1177\/0018720819875376","DOI":"10.1177\/0018720819875376"},{"key":"e_1_2_1_9_1","doi-asserted-by":"crossref","unstructured":"M. R. Endsley and D. J. Garland. 2000. Situation awareness: Analysis and measurement M. Endsley and D. J. Garland (Eds.). CRC Press Boca Raton FL.  M. R. Endsley and D. J. Garland. 2000. Situation awareness: Analysis and measurement M. Endsley and D. J. Garland (Eds.). CRC Press Boca Raton FL.","DOI":"10.1201\/b12461"},{"key":"e_1_2_1_10_1","unstructured":"M. R. Endsley and D. G. Jones. 2012. Designing for situation awareness: An approach to human-centered design (2nd ed.). Taylor & Francis London.  M. R. Endsley and D. G. Jones. 2012. Designing for situation awareness: An approach to human-centered design (2nd ed.). Taylor & Francis London."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1518\/001872095779064555"},{"key":"e_1_2_1_12_1","first-page":"75300H","volume-title":"Visualization and Data Analysis","volume":"7530","author":"Erbacher R. F.","unstructured":"R. F. Erbacher , D. A. Frincke , P. C. Wong , S. Moody , and G. Fink . 2010. Cognitive task analysis of network analysts and managers for network situational awareness . In Visualization and Data Analysis , Vol. 7530 . International Society for Optics and Photonics , p. 75300H . R. F. Erbacher, D. A. Frincke, P. C. Wong, S. Moody, and G. Fink. 2010. Cognitive task analysis of network analysts and managers for network situational awareness. In Visualization and Data Analysis, Vol. 7530. International Society for Optics and Photonics, p. 75300H."},{"key":"e_1_2_1_13_1","unstructured":"J. Freund and J. Jones. 2014. Measuring and Managing Information Risk: A FAIR Approach. Butterworth-Heinemann Oxford UK.  J. Freund and J. Jones. 2014. Measuring and Managing Information Risk: A FAIR Approach. Butterworth-Heinemann Oxford UK."},{"key":"e_1_2_1_14_1","volume-title":"Proceedings of the IEEE ISI Conference. 284--288","author":"Giacobe N. A.","unstructured":"N. A. Giacobe , M. D. McNeese , V. F. Mancuso , and D. Minotra . 2013. Capturing human cognition in cyber-security simulations with NETS . In Proceedings of the IEEE ISI Conference. 284--288 . N. A. Giacobe, M. D. McNeese, V. F. Mancuso, and D. Minotra. 2013. Capturing human cognition in cyber-security simulations with NETS. In Proceedings of the IEEE ISI Conference. 284--288."},{"key":"e_1_2_1_15_1","volume-title":"Situation Awareness in Defensive Cyberspace Operations: An Annotated Bibliographic Assessment Through","author":"Gutzwiller R. S.","year":"2015","unstructured":"R. S. Gutzwiller . 2019. Situation Awareness in Defensive Cyberspace Operations: An Annotated Bibliographic Assessment Through 2015 . (No. TR-3184). NIWC Pacific San Diego United States . R. S. Gutzwiller. 2019. Situation Awareness in Defensive Cyberspace Operations: An Annotated Bibliographic Assessment Through 2015. (No. TR-3184). NIWC Pacific San Diego United States."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1177\/1555343412451749"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 322--326","author":"Gutzwiller R. S.","year":"1931","unstructured":"R. S. Gutzwiller , S. Fugate , B. D. Sawyer , and P. A. Hancock . 2015. The human factors of cyber network defense . In Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 322--326 . https:\/\/doi.org\/10.1177\/154 1931 215591067 R. S. Gutzwiller, S. Fugate, B. D. Sawyer, and P. A. Hancock. 2015. The human factors of cyber network defense. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 322--326. https:\/\/doi.org\/10.1177\/1541931215591067"},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA\u201916)","author":"Gutzwiller R. S.","year":"2016","unstructured":"R. S. Gutzwiller , S. M. Hunt , and D. S. Lange . 2016. A task analysis toward characterizing cyber-cognitive situation awareness (CCSA) in cyber defense analysts . In Proceedings of the IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA\u201916) . 14--20. https:\/\/doi.org\/10.1109\/COGSIMA. 2016 .7497780 R. S. Gutzwiller, S. M. Hunt, and D. S. Lange. 2016. A task analysis toward characterizing cyber-cognitive situation awareness (CCSA) in cyber defense analysts. In Proceedings of the IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA\u201916). 14--20. https:\/\/doi.org\/10.1109\/COGSIMA.2016.7497780"},{"key":"e_1_2_1_19_1","volume-title":"Situation awareness: Analysis and measurement","author":"Jones D.","unstructured":"D. Jones . 2000. Subjective measures of situation awareness . In Situation awareness: Analysis and measurement , M. R. Endsley and D. Garland (Eds.). Lawrence Erlbaum Associates , Mahwah, NJ , 113--128. D. Jones. 2000. Subjective measures of situation awareness. In Situation awareness: Analysis and measurement, M. R. Endsley and D. Garland (Eds.). Lawrence Erlbaum Associates, Mahwah, NJ, 113--128."},{"key":"e_1_2_1_20_1","first-page":"507","article-title":"Sources of situation awareness errors in aviation","volume":"67","author":"Jones D. G.","year":"1996","unstructured":"D. G. Jones , and M. R. Endsley . 1996 . Sources of situation awareness errors in aviation . Aviat. Space Environ. Med. 67 , 6 (1996), 507 -- 512 . https:\/\/doi.org\/10.1039\/c4qo00187g D. G. Jones, and M. R. Endsley. 1996. Sources of situation awareness errors in aviation. Aviat. Space Environ. Med. 67, 6 (1996), 507--512. https:\/\/doi.org\/10.1039\/c4qo00187g","journal-title":"Aviat. Space Environ. Med."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1080\/135467897394329"},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 279--283","author":"Mahoney S.","unstructured":"S. Mahoney , E. Roth , K. Steinke , J. Pfautz , C. Wu , and M. Farry . 2010. A cognitive task analysis for cyber situational awareness . In Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 279--283 . S. Mahoney, E. Roth, K. Steinke, J. Pfautz, C. Wu, and M. Farry. 2010. A cognitive task analysis for cyber situational awareness. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 279--283."},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the IEEE International Conference on Information Technology: New Generations. 937--942","author":"Malviya A.","year":"2011","unstructured":"A. Malviya , G. A. Fink , L. Sego , and B. Endicott-Popovsky . 2011. Situational awareness as a measure of performance in cyber security collaborative work . In Proceedings of the IEEE International Conference on Information Technology: New Generations. 937--942 . https:\/\/doi.org\/10.1109\/ITNG. 2011 .161 A. Malviya, G. A. Fink, L. Sego, and B. Endicott-Popovsky. 2011. Situational awareness as a measure of performance in cyber security collaborative work. In Proceedings of the IEEE International Conference on Information Technology: New Generations. 937--942. https:\/\/doi.org\/10.1109\/ITNG.2011.161"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the IEEE Symposium on Visualization for Cyber Security (VizSec\u201915)","author":"Mckenna S.","year":"2015","unstructured":"S. Mckenna , D. Staheli , and M. Meyer . 2015. Unlocking user-centered design methods for building cyber security visualizations . In Proceedings of the IEEE Symposium on Visualization for Cyber Security (VizSec\u201915) . 1--8. https:\/\/doi.org\/10.1109\/VIZSEC. 2015 .7312771 S. Mckenna, D. Staheli, and M. Meyer. 2015. Unlocking user-centered design methods for building cyber security visualizations. In Proceedings of the IEEE Symposium on Visualization for Cyber Security (VizSec\u201915). 1--8. https:\/\/doi.org\/10.1109\/VIZSEC.2015.7312771"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1080\/00140130701276640"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1177\/0018720813501549"},{"key":"e_1_2_1_27_1","first-page":"1","article-title":"Understanding operator fatigue, frustration, and cognitive workload in tactical cybersecurity operations","volume":"16","author":"Paul C. L.","year":"2017","unstructured":"C. L. Paul , and J. Dykstra . 2017 . Understanding operator fatigue, frustration, and cognitive workload in tactical cybersecurity operations . J. Info. Warfare 16 , 2 (2017), 1 - 11 . C. L. Paul, and J. Dykstra. 2017. Understanding operator fatigue, frustration, and cognitive workload in tactical cybersecurity operations. J. Info. Warfare 16, 2 (2017), 1-11.","journal-title":"J. Info. Warfare"},{"key":"e_1_2_1_28_1","doi-asserted-by":"crossref","first-page":"626","DOI":"10.1177\/0018720818769249","article-title":"Information-pooling bias in collaborative security incident correlation analysis","volume":"60","author":"Rajivan P.","year":"2018","unstructured":"P. Rajivan and N. J. Cooke . 2018 . Information-pooling bias in collaborative security incident correlation analysis . Hum. Fact. 60 , 5 (2018), 626 -- 639 . https:\/\/doi.org\/10.1177\/0018720818769249 P. Rajivan and N. J. Cooke. 2018. Information-pooling bias in collaborative security incident correlation analysis. Hum. Fact. 60, 5 (2018), 626--639. https:\/\/doi.org\/10.1177\/0018720818769249","journal-title":"Hum. Fact."},{"key":"e_1_2_1_29_1","unstructured":"P. M. Salmon N. A. Stanton G. H. Walker and D. P. Jenkins. 2009. Distributed Situation Awareness: Theory Measurement and Application to Teamwork. CRC Press Boca Raton FL.  P. M. Salmon N. A. Stanton G. H. Walker and D. P. Jenkins. 2009. Distributed Situation Awareness: Theory Measurement and Application to Teamwork. CRC Press Boca Raton FL."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1518\/hfes.46.3.461.50392"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the 11th Workshop on Visualization for Cyber Security, 49--56","author":"Staheli D.","unstructured":"D. Staheli , T. Yu , R. Crouser , S. Damodaran , K. Nam , D. O'Gwynn , and L. Harrison . 2014. Visualization evaluation for cyber security: Trends and future directions . In Proceedings of the 11th Workshop on Visualization for Cyber Security, 49--56 . D. Staheli, T. Yu, R. Crouser, S. Damodaran, K. Nam, D. O'Gwynn, and L. Harrison. 2014. Visualization evaluation for cyber security: Trends and future directions. In Proceedings of the 11th Workshop on Visualization for Cyber Security, 49--56."},{"key":"e_1_2_1_32_1","doi-asserted-by":"crossref","unstructured":"S. Stevens-Adams A. Carbajal A. Silva K. Nauer B. Anderson T. Reed and C. Forsythe. 2011. Enhanced training for cyber situational awareness. In Foundations of Augmented Cognition. Springer Berlin 90--99.  S. Stevens-Adams A. Carbajal A. Silva K. Nauer B. Anderson T. Reed and C. Forsythe. 2011. Enhanced training for cyber situational awareness. In Foundations of Augmented Cognition. Springer Berlin 90--99.","DOI":"10.1007\/978-3-642-39454-6_10"},{"key":"e_1_2_1_33_1","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1080\/10508414.2011.556492","article-title":"Prediction in situation awareness: Confidence bias and underlying cognitive abilities","volume":"21","author":"Sulistyawati K.","year":"2011","unstructured":"K. Sulistyawati , C. D. Wickens , and Y. P. Chui . 2011 . Prediction in situation awareness: Confidence bias and underlying cognitive abilities . Int. J. Aviation Psychol. 21 , 2 (2011), 153 -- 174 . https:\/\/doi.org\/10.1080\/10508414.2011.556492 K. Sulistyawati, C. D. Wickens, and Y. P. Chui. 2011. Prediction in situation awareness: Confidence bias and underlying cognitive abilities. Int. J. Aviation Psychol. 21, 2 (2011), 153--174. https:\/\/doi.org\/10.1080\/10508414.2011.556492","journal-title":"Int. J. Aviation Psychol."},{"key":"e_1_2_1_34_1","first-page":"23","article-title":"Situational Awareness Rating Technique (SART): The development of a tool for aircrew systems design","volume":"17","author":"Taylor R. M.","year":"1990","unstructured":"R. M. Taylor . 1990 . Situational Awareness Rating Technique (SART): The development of a tool for aircrew systems design . AGARD Situation. Aware. Aerospace Operat. 17 (1990), 23 -- 53 . R. M. Taylor. 1990. Situational Awareness Rating Technique (SART): The development of a tool for aircrew systems design. AGARD Situation. Aware. Aerospace Operat. 17 (1990), 23--53.","journal-title":"AGARD Situation. Aware. Aerospace Operat."},{"key":"e_1_2_1_35_1","first-page":"125","article-title":"Modelling the cognitive work of Cyber Protection Teams","volume":"4","author":"Trent S.","year":"2019","unstructured":"S. Trent , R. R. Hoffman , D. Merritt , and S. Smith . 2019 . Modelling the cognitive work of Cyber Protection Teams . Cyber Defense Rev. 4 , 1 (2019), 125 -- 135 . S. Trent, R. R. Hoffman, D. Merritt, and S. Smith. 2019. Modelling the cognitive work of Cyber Protection Teams. Cyber Defense Rev. 4, 1 (2019), 125--135.","journal-title":"Cyber Defense Rev."},{"key":"e_1_2_1_36_1","volume-title":"Proceedings of International Workshop on Security and Privacy Analytics (IWSPA\u201915)","author":"Zhang H.","unstructured":"H. Zhang , S. Maoyuan , D. Yao , and C. North . 2015. Visualizing traffic causality for analyzing network anomalies . In Proceedings of International Workshop on Security and Privacy Analytics (IWSPA\u201915) . 37--42. https:\/\/doi.org\/10.1145\/2713579.2713583. H. Zhang, S. Maoyuan, D. Yao, and C. North. 2015. Visualizing traffic causality for analyzing network anomalies. In Proceedings of International Workshop on Security and Privacy Analytics (IWSPA\u201915). 37--42. https:\/\/doi.org\/10.1145\/2713579.2713583."}],"container-title":["Digital Threats: Research and Practice"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3384471","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3384471","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:33:12Z","timestamp":1750199592000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3384471"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,2]]},"references-count":36,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2020,9,30]]}},"alternative-id":["10.1145\/3384471"],"URL":"https:\/\/doi.org\/10.1145\/3384471","relation":{},"ISSN":["2692-1626","2576-5337"],"issn-type":[{"value":"2692-1626","type":"print"},{"value":"2576-5337","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,2]]},"assertion":[{"value":"2019-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-02-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-09-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}