{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:21:26Z","timestamp":1760242886733,"version":"build-2065373602"},"reference-count":36,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2016,9,21]],"date-time":"2016-09-21T00:00:00Z","timestamp":1474416000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100006602","name":"Air Force Research Laboratory","doi-asserted-by":"publisher","award":["FA8650-12-M-6266"],"award-info":[{"award-number":["FA8650-12-M-6266"]}],"id":[{"id":"10.13039\/100006602","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>Vehicles and platforms with multiple sensors connect people in multiple roles with different responsibilities to scenes of interest. For many of these human\u2013sensor systems there are a variety of algorithms that transform, select, and filter the sensor data prior to human intervention. Emergency response, precision agriculture, and intelligence, surveillance and reconnaissance (ISR) are examples of these human\u2013computation\u2013sensor systems. The authors examined a case of the latter to understand how people in various roles utilize the algorithms output to identify meaningful properties in data streams given uncertainty. The investigations revealed: (a) that increasingly complex interactions occur across agents in the human\u2013computation\u2013sensor system; and (b) analysts struggling to interpret the output of \u201cblack box\u201d algorithms given uncertainty and change in the scenes of interest. The paper presents a new interactive visualization concept designed to \u201copen up the black box\u201d of sensor processing algorithms to support human analysts as they look for meaning in feeds from sensors.<\/jats:p>","DOI":"10.3390\/informatics3030016","type":"journal-article","created":{"date-parts":[[2016,9,21]],"date-time":"2016-09-21T10:11:26Z","timestamp":1474452686000},"page":"16","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Opening up the Black Box of Sensor Processing Algorithms through New Visualizations"],"prefix":"10.3390","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2537-2241","authenticated-orcid":false,"given":"Alexander","family":"Morison","sequence":"first","affiliation":[{"name":"Department of Integrated Systems Engineering, The Ohio State University, Columbus, OH 45420, USA"}]},{"given":"David","family":"Woods","sequence":"additional","affiliation":[{"name":"Department of Integrated Systems Engineering, The Ohio State University, Columbus, OH 45420, USA"}]}],"member":"1968","published-online":{"date-parts":[[2016,9,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Hoffman, R., Hancock, P., Parasuraman, R., Szalma, J., and Scerbo, M. (2015). Cambridge Handbook of Applied Perception Research, Cambridge University Press.","DOI":"10.1017\/CBO9780511973017"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kaisler, S., Armour, F., Espinosa, J.A., and Money, W. (2013, January 7\u201310). Big Data: Issues and Challenges Moving Forward. Proceedings of the 2013 46th Hawaii International Conference on System Sciences (HICSS), Maui, HI, USA.","DOI":"10.1109\/HICSS.2013.645"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1007\/s101110200002","article-title":"Can We Ever Escape from Data Overload? A Cognitive Systems Diagnosis","volume":"4","author":"Woods","year":"2002","journal-title":"Cogn. Technol. Work"},{"key":"ref_4","unstructured":"Sonka, M., Hlavac, V., and Boyle, R. (2014). Image Processing, Analysis, and Machine Vision, Cengage Learning."},{"key":"ref_5","unstructured":"Jensen, J.R. (1986). Introductory Digital Image Processing: A Remote Sensing Perspective, Prentice Hall, Inc."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1523","DOI":"10.1002\/asi.23294","article-title":"Big Data, Bigger Dilemmas: A Critical Review","volume":"66","author":"Ekbia","year":"2015","journal-title":"J. Assoc. Inf. Sci. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1109\/TGRS.2003.819189","article-title":"Estimation of Number of Spectrally Distinct Signal Sources in Hyperspectral Imagery","volume":"42","author":"Chang","year":"2004","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1207\/s15327051hci0101_2","article-title":"Systems with Human Monitors: A Signal Detection Analysis","volume":"1","author":"Sorkin","year":"1985","journal-title":"Hum.-Comput. Interact."},{"key":"ref_9","unstructured":"Murphy, R., and Shields, J. The Role of Autonomy in DoD Systems. Available online: https:\/\/fas.org\/irp\/agency\/dod\/dsb\/autonomy.pdf."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1109\/3468.568744","article-title":"Brittleness in the Design of Cooperative Problem-Solving Systems: The Effects on User Performance","volume":"27","author":"Smith","year":"1997","journal-title":"IEEE Trans. Syst. Man Cybern. Part Syst. Hum."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1109\/MIS.2004.74","article-title":"Ten Challenges for Making Automation a \u201cTeam Player\u201d in Joint Human-Agent Activity","volume":"19","author":"Klein","year":"2004","journal-title":"IEEE Intell. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Crandall, B., Klein, G.A., and Hoffman, R.R. (2006). Working Minds: A Practitioner\u2019s Guide to Cognitive Task Analysis, MIT Press.","DOI":"10.7551\/mitpress\/7304.001.0001"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/79.974715","article-title":"Signal Processing for Hyperspectral Image Exploitation","volume":"19","author":"Shaw","year":"2002","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_14","unstructured":"Cook, K.A., and Thomas, J.J. (2005). Illuminating the Path: The Research and Development Agenda for Visual Analytics."},{"key":"ref_15","unstructured":"Keim, D., Andrienko, G., Fekete, J.-D., G\u00f6rg, C., Kohlhammer, J., and Melan\u00e7on, G. (2008). Information Visualization, Springer."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Hutchins, E. (1995). Cognition in the Wild, The MIT Press.","DOI":"10.7551\/mitpress\/1881.001.0001"},{"key":"ref_17","unstructured":"De Keyser, V. (1992). Design for Manufacturability: A Systems Approach to Concurrent Engineering in Ergonomics, Taylor & Francis."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1037\/h0061470","article-title":"The Critical Incident Technique","volume":"51","author":"Flanagan","year":"1954","journal-title":"Psychol. Bull."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1006\/obhd.1995.1039","article-title":"Eliciting Knowledge from Experts: A Methodological Analysis","volume":"62","author":"Hoffman","year":"1995","journal-title":"Organ. Behav. Hum. Decis. Process."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1177\/155534340700100104","article-title":"Challenges for Cognition in Intelligence Analysis","volume":"1","author":"Trent","year":"2007","journal-title":"J. Cogn. Eng. Decis. Mak."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1080\/01402398408437190","article-title":"Intelligence and the Problem of Strategic Surprise","volume":"7","author":"Handel","year":"1984","journal-title":"J. Strateg. Stud."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1016\/S1364-6613(00)01751-4","article-title":"Mental Models and Deduction","volume":"5","year":"2001","journal-title":"Trends Cogn. Sci."},{"key":"ref_23","first-page":"7","article-title":"Some Observations on Mental Models","volume":"7","author":"Norman","year":"1983","journal-title":"Ment. Models"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/BF01619263","article-title":"Evaluating the Human Engineering of Microprocessor-Controlled Operating Room Devices","volume":"7","author":"Cook","year":"1991","journal-title":"J. Clin. Monit."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1016\/S0020-7373(84)80043-7","article-title":"Visual Momentum: A Concept to Improve the Cognitive Coupling of Person and Computer","volume":"21","author":"Woods","year":"1984","journal-title":"Int. J. Man-Mach. Stud."},{"key":"ref_26","unstructured":"Yuhas, R.H., Goetz, A.F., and Boardman, J.W. (1992, January 1\u20135). Discrimination among Semi-Arid Landscape Endmembers Using the Spectral Angle Mapper (SAM) Algorithm. Proceedings of Summaries 3rd Annual JPL Airborne Geoscience Workshop, Pasadena, CA, USA."},{"key":"ref_27","unstructured":"Clark, R.N., Swayze, G., Boardman, J., and Kruse, F. Comparison of Three Methods for Materials Identification and Mapping with Imaging Spectroscopy, 1993, Available online: http:\/\/ntrs.nasa.gov\/search.jsp?R=19950017433."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"779","DOI":"10.1109\/36.298007","article-title":"Hyperspectral Image Classification and Dimensionality Reduction: An Orthogonal Subspace Projection Approach","volume":"32","author":"Harsanyi","year":"1994","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.jag.2005.06.001","article-title":"The Effectiveness of Spectral Similarity Measures for the Analysis of Hyperspectral Imagery","volume":"8","year":"2006","journal-title":"Int. J. Appl. Earth Obs. Geoinformation"},{"key":"ref_30","unstructured":"Card, S.K., Mackinlay, J.D., and Shneiderman, B. (1999). Readings in Information Visualization: Using Vision to Think, Morgan Kaufmann."},{"key":"ref_31","unstructured":"Inselberg, A., and Dimsdale, B. (1991). Human-Machine Interactive Systems, Plenum Press."},{"key":"ref_32","first-page":"95","article-title":"State of the Art of Parallel Coordinates","volume":"2013","author":"Heinrich","year":"2013","journal-title":"STAR Proc. Eurograph."},{"key":"ref_33","unstructured":"Ware, C. (2012). Information Visualization: Perception for Design, Morgan Kaufmann. [3rd ed.]."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1177\/001872089303500105","article-title":"Emergent Features and Graphical Elements\u2014Designing More Effective Configural Displays","volume":"35","author":"Bennett","year":"1993","journal-title":"Hum. Factors"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1177\/001872089203400502","article-title":"Graphical Displays: Implications for Divided Attention, Focused Attention, and Problem Solving","volume":"34","author":"Bennett","year":"1992","journal-title":"Hum. Factors"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1207\/s15516709cog1703_3","article-title":"Concept Discovery in a Scientific Domain","volume":"17","author":"Dunbar","year":"1993","journal-title":"Cogn. Sci."}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/3\/3\/16\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T19:31:24Z","timestamp":1760211084000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/3\/3\/16"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,9,21]]},"references-count":36,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2016,9]]}},"alternative-id":["informatics3030016"],"URL":"https:\/\/doi.org\/10.3390\/informatics3030016","relation":{},"ISSN":["2227-9709"],"issn-type":[{"type":"electronic","value":"2227-9709"}],"subject":[],"published":{"date-parts":[[2016,9,21]]}}}