{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,17]],"date-time":"2025-11-17T05:55:54Z","timestamp":1763358954492,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2023,11,29]],"date-time":"2023-11-29T00:00:00Z","timestamp":1701216000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>This research addresses the data overload faced by intelligence searchers in government and defense agencies. The study leverages methods from the Cognitive Systems Engineering (CSE) literature to generate insights into the intelligence search work domain. These insights are applied to a supporting concept and requirements for designing and evaluating a human-AI agent team specifically for intelligence search tasks. Domain analysis reveals the dynamic nature of the \u2018value structure\u2019, a term that describes the evolving set of criteria governing the intelligence search process. Additionally, domain insight provides details for search aggregation and conceptual spaces from which the value structure could be efficiently applied for intelligence search. Support system designs that leverage these findings may enable an intelligence searcher to interact with and understand data at more abstract levels to improve task efficiency. Additionally, new system designs can support the searcher by facilitating an \u2018Ambient Awareness\u2019 of non-selected objects in a large data field through relevant system cues. Ambient Awareness achieved through the supporting concept and AI teaming has the potential to address the data overload problem while increasing the breadth and depth of search coverage.<\/jats:p>","DOI":"10.3390\/systems11120561","type":"journal-article","created":{"date-parts":[[2023,11,29]],"date-time":"2023-11-29T03:53:52Z","timestamp":1701230032000},"page":"561","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Analysis and Requirement Generation for Defense Intelligence Search: Addressing Data Overload through Human\u2013AI Agent System Design for Ambient Awareness"],"prefix":"10.3390","volume":"11","author":[{"given":"Mark C.","family":"Duncan","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, Air Force Institute of Technology, 2950 Hobson Way (AFIT\/ENG), Wright Patterson AFB, Dayton, OH 45433, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6306-3437","authenticated-orcid":false,"given":"Michael E.","family":"Miller","sequence":"additional","affiliation":[{"name":"Department of Systems Engineering and Management, Air Force Institute of Technology, 2950 Hobson Way (AFIT\/ENV), Wright Patterson AFB, Dayton, OH 45433, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4982-9859","authenticated-orcid":false,"given":"Brett J.","family":"Borghetti","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Air Force Institute of Technology, 2950 Hobson Way (AFIT\/ENG), Wright Patterson AFB, Dayton, OH 45433, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,11,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Patterson, E.S., Woods, D.D., Tinapple, D., Roth, E.M., Finley, J.M., and Kuperman, G.G. (2001). Aiding the Intelligence Analyst in Situations of Data Overload: From Problem Definition to Design Concept Exploration, Human Effectiveness Directorate. Institute for Ergonomics\/Cognitive Systems Engineering Laboratory Report, ERGO-CSEL.","DOI":"10.21236\/ADA397621"},{"key":"ref_2","unstructured":"Resilient Cognitive Solutions (2021). The Brittleness Reference Guide, Resilient Cognitive Solutions."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1109\/21.156574","article-title":"Ecological interface design: Theoretical foundations","volume":"22","author":"Vicente","year":"1992","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1109\/TSMC.1985.6313353","article-title":"The role of hierarchical knowledge representation in decision making and system management","volume":"SMC-15","author":"Rasmussen","year":"1985","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1207\/S15327566IJCE0503_13","article-title":"Model-Based Approaches for Analyzing Cognitive Work: A Comparison of Abstraction Hierarchy, Multilevel Flow Modeling, and Decision Ladder Modeling","volume":"5","author":"Burns","year":"2001","journal-title":"Int. J. Cogn. Ergon."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1007\/s10111-002-0109-4","article-title":"Making sense of the abstraction hierarchy in the power plant domain","volume":"5","author":"Lind","year":"2003","journal-title":"Cogn. Technol. Work"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1080\/14639221003602473","article-title":"Analysis of complex team-based systems: Augmentations to goal-directed task analysis and cognitive work analysis","volume":"12","author":"Humphrey","year":"2011","journal-title":"Theor. Issues Ergon. Sci."},{"key":"ref_8","unstructured":"Endsley, M.R., and Jones, D.G. (2012). Designing for Situation Awareness: An Approach to User-Centered Design, CRC Press, Inc."},{"key":"ref_9","unstructured":"Elm, W.C., Potter, S.S., Gualtieri, J.W., Easter, J.R., and Roth, E.M. (2003). Handbook of Cognitive Task Design, CRC Press."},{"key":"ref_10","unstructured":"Potter, S.S., Elm, W.C., Roth, E.M., Gualtieri, J., and Easter, J. (2002). State of the Art Report (SOAR): Cognitive Systems Engineering in Military Aviation Environments: Avoiding Cogminutia Fragmentosa, Human Systems Information Analysis Center."},{"key":"ref_11","unstructured":"Potter, S.S., Gualtieri, J., and Elm, W.C. (2003). Handbook for Cognitive Task Design, CRC Press."},{"key":"ref_12","unstructured":"Rasmussen, J., Pejtersen, A.M., and Goodstein, L.P. (1994). Cognitive Systems Engineering, Wiley."},{"key":"ref_13","unstructured":"Rasmussen, J. (1986). Information Processing and Human-Machine Interaction: An Approach to Cognitive Engineering, Elsevier Science Inc."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1177\/0018720819835990","article-title":"Ecological Interface Design: Thirty-Plus Years of Refinement, Progress, and Potential","volume":"61","author":"Bennett","year":"2019","journal-title":"Hum. Factors J. Hum. Factors Ergon. Soc."},{"key":"ref_15","unstructured":"Marr, D. (1982). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information, W.H. Freeman."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Bennett, K.B., and Flach, J.M. (2011). Display and Interface Design: Subtle Science, Exact Art, CRC Press.","DOI":"10.1201\/b10774"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1207\/s15326969eco0203_2","article-title":"The Ecology of Human-Machine Systems II: Mediating \u201cDirect Perception\u201d in Complex Work Domains","volume":"2","author":"Vicente","year":"1990","journal-title":"Ecol. Psychol."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kane, A.A., Paletz, S.B.F., Vahlkamp, S.H., Nelson, T., Porter, A., Diep, M., and Carraway, M. (2023, January 23\u201327). Intelligence Analysis Shift Work: Sensemaking Processes, Tensions, and Takeaways. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Washington, DC, USA.","DOI":"10.1177\/21695067231192569"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"753","DOI":"10.1177\/1745691615598511","article-title":"Improving Intelligence Analysis with Decision Science","volume":"10","author":"Dhami","year":"2015","journal-title":"Perspect. Psychol. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1177\/154193120504900318","article-title":"Finding Decision Support Requirements for Effective Intelligence Analysis Tools","volume":"49","author":"Elm","year":"2005","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"390","DOI":"10.1177\/1541931214581080","article-title":"Supporting Representation Management in Intelligence Analysis through Automated Decision Aids","volume":"58","author":"Voshell","year":"2014","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1177\/154193129904300310","article-title":"Can We Ever Escape from Data Overload? A Cognitive Systems Diagnosis","volume":"43","author":"Woods","year":"1999","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1177\/154193120104500437","article-title":"Using Cognitive Task Analysis (CTA) to Seed Design Concepts for Intelligence Analysts Under Data Overload","volume":"45","author":"Patterson","year":"2001","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet."},{"key":"ref_24","unstructured":"Grupen, N. (2022, March 08). AlphaFold, GPT-3 and How to Augment Intelligence with AI|Future. Available online: https:\/\/future.com\/alphafold-gpt-3-and-how-to-augment-intelligence-with-ai\/."},{"key":"ref_25","unstructured":"Trilog Group (2023, August 18). Social Business Applications Promote Observability via Ambient Awareness. Available online: https:\/\/theprojectwall.wordpress.com\/2011\/02\/24\/social-business-applications-promote-observability-via-ambient-awareness\/."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Madni, A.M., and Madni, C.C. (2018). Architectural Framework for Exploring Adaptive Human-Machine Teaming Options in Simulated Dynamic Environments. Systems, 6.","DOI":"10.3390\/systems6040044"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Schadd, M.P.D., Schoonderwoerd, T.A.J., van den Bosch, K., Visker, O.H., Haije, T., and Veltman, K.H.J. (2022). \u201cI\u2019m Afraid I Can\u2019t Do That, Dave\u201d; Getting to Know Your Buddies in a Human-Agent Team. Systems, 10.","DOI":"10.3390\/systems10010015"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Johnson, C.D., Miller, M.E., Rusnock, C.F., and Jacques, D.R. (2020). Applying Control Abstraction to the Design of Human\u2013Agent Teams. Systems, 8.","DOI":"10.3390\/systems8020010"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1177\/1541931215591027","article-title":"The Human Factors of Intelligence Analysis","volume":"59","author":"McNeese","year":"2015","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Crandall, B., Klein, G.A., and Hoffman, R.R. (2006). Working Minds: A Practitioner\u2019s Guide to Cognitive Task Analysis, MIT Press.","DOI":"10.7551\/mitpress\/7304.001.0001"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"254","DOI":"10.1518\/001872098779480442","article-title":"Use of the Critical Decision Method to Elicit Expert Knowledge: A Case Study in the Methodology of Cognitive Task Analysis","volume":"40","author":"Hoffman","year":"1998","journal-title":"Hum. Factors J. Hum. Factors Ergon. Soc."},{"key":"ref_32","unstructured":"Hitchins, D.K. (1992). Putting Systems to Work, John Wiley & Sons."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Vicente, K.J. (1999). Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work, CRC Press.","DOI":"10.1201\/b12457"}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/11\/12\/561\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:33:12Z","timestamp":1760131992000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/11\/12\/561"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,29]]},"references-count":33,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["systems11120561"],"URL":"https:\/\/doi.org\/10.3390\/systems11120561","relation":{},"ISSN":["2079-8954"],"issn-type":[{"type":"electronic","value":"2079-8954"}],"subject":[],"published":{"date-parts":[[2023,11,29]]}}}