{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T11:49:19Z","timestamp":1753876159818,"version":"3.41.2"},"reference-count":39,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2024,8,29]],"date-time":"2024-08-29T00:00:00Z","timestamp":1724889600000},"content-version":"vor","delay-in-days":241,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,1,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>In this human subjects study, we sought to enable user-based detection of malware within portable document format (PDF) files. Such malware is often difficult to detect with traditional malware detection tools. Humans are excellent sensors and pattern detectors and could be a key to more robust malware detection but need something to sense. By design, malware is often hidden deep within a file and its presence or operation may be completely hidden from users. To combat this, we built a visualization to expose underlying file contents and conducted a study to assess whether the visualization would enable novice users to detect malware embedded within PDF files. We found that when users engaged with the tool, detection for PDF malware was well above chance performance, better than a control condition, and with high discriminability. The display significantly improved user detection of malware in PDF files; combined with feedback and the ability to provide aggregated detection information to security analysts in a future version, we believe it could enable more effective detection and response. This research highlights the need for integration as well as experimentation between human and machine to best improve cyber defense.<\/jats:p>","DOI":"10.1093\/cybsec\/tyae016","type":"journal-article","created":{"date-parts":[[2024,8,29]],"date-time":"2024-08-29T12:16:45Z","timestamp":1724933805000},"source":"Crossref","is-referenced-by-count":0,"title":["A novel visual interface enables human detection of malware in portable document format"],"prefix":"10.1093","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2159-8131","authenticated-orcid":false,"given":"Robert S","family":"Gutzwiller","sequence":"first","affiliation":[{"name":"Fulton Schools of Engineering, Arizona State University , Mesa, Arizona, 85201 ,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6733-3219","authenticated-orcid":false,"given":"Sunny J","family":"Fugate","sequence":"additional","affiliation":[{"name":"Naval Information Warfare Center Pacific , San Diego, California, 92106 ,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6955-1965","authenticated-orcid":false,"given":"Jamie R","family":"Lukos","sequence":"additional","affiliation":[{"name":"Naval Information Warfare Center Pacific , San Diego, California, 92106 ,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3820-8678","authenticated-orcid":false,"given":"Karl","family":"Wiegand","sequence":"additional","affiliation":[{"name":"Naval Information Warfare Center Pacific , San Diego, California, 92106 ,","place":["United States"]}]}],"member":"286","published-online":{"date-parts":[[2024,8,29]]},"reference":[{"key":"2024121909133810800_bib1","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1177\/1541931215591067","article-title":"The human factors of cyber network defense","volume":"59","author":"Gutzwiller","year":"2015","journal-title":"Proc Hum Factors Ergon Soc Annu Meet"},{"year":"1983","author":"Carroll","key":"2024121909133810800_bib2"},{"key":"2024121909133810800_bib3","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/MC.1985.1663001","article-title":"Abstractions for user interface design","volume":"9","author":"Coutaz","year":"1985","journal-title":"Computer"},{"key":"2024121909133810800_bib4","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1504\/IJSN.2011.043671","article-title":"Hidden information in Microsoft Word","volume":"6","author":"Liu","year":"2011","journal-title":"Int J Secur Netw"},{"key":"2024121909133810800_bib5","first-page":"326","article-title":"A phishing vulnerability analysis of web based systems","volume-title":"Proceedings of the IEEE Symposium on Computers and Communications","author":"Yu","year":"2008"},{"key":"2024121909133810800_bib6","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1109\/MSP.2011.98","article-title":"Obfuscation: the hidden malware","volume":"9","author":"O\u2019Kane","year":"2011","journal-title":"IEEE Secur Priv"},{"key":"2024121909133810800_bib7","first-page":"297","article-title":"Malware obfuscation techniques: a brief survey","volume-title":"Proceedings of the International Conference on Broadband and Wireless Computing, Communication and Applications","author":"You","year":"2010"},{"key":"2024121909133810800_bib8","doi-asserted-by":"crossref","first-page":"800","DOI":"10.1145\/358198.358218","article-title":"Training wheels in a user interface","volume":"27","author":"Carroll","year":"1984","journal-title":"Commun ACM"},{"key":"2024121909133810800_bib9","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1109\/2.955105","article-title":"Supporting usability through software architecture","volume":"34","author":"Bass","year":"2001","journal-title":"Computer"},{"year":"2004","author":"Whitten","key":"2024121909133810800_bib10"},{"volume-title":"Security and Usability: Designing Secure Systems That People Can Use","year":"2005","author":"Cranor","key":"2024121909133810800_bib11"},{"key":"2024121909133810800_bib12","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1023\/A:1011902718709","article-title":"Transforming the \u201cweakest link\u201d\u2014a human\/computer interaction approach to usable and effective security","volume":"19","author":"Sasse","year":"2001","journal-title":"BT Technol J"},{"key":"2024121909133810800_bib13","first-page":"169","article-title":"Why Johnny can't encrypt: a usability evaluation of PGP 5.0","volume":"348","author":"Whitten","year":"1999","journal-title":"USENIX Secur Symp"},{"key":"2024121909133810800_bib14","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1518\/001872008X312198","article-title":"Humans: still vital after all these years of automation","volume":"50","author":"Parasuraman","year":"2008","journal-title":"Hum Factors"},{"key":"2024121909133810800_bib15","doi-asserted-by":"crossref","first-page":"775","DOI":"10.1016\/0005-1098(83)90046-8","article-title":"Ironies of automation","volume":"19","author":"Bainbridge","year":"1983","journal-title":"Automatica"},{"key":"2024121909133810800_bib16","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1109\/THMS.2017.2732506","article-title":"Ironies of automation: still unresolved after all these years","volume":"48","author":"Strauch","year":"2017","journal-title":"IEEE Trans Human\u2013Machine Syst"},{"key":"2024121909133810800_bib17","doi-asserted-by":"crossref","first-page":"1968","DOI":"10.24251\/HICSS.2021.241","article-title":"Human factors in automating cyber operations","volume-title":"Proceedings of the 54th Hawaii International Conference on System Sciences","author":"Gutzwiller","year":"2021"},{"key":"2024121909133810800_bib18","doi-asserted-by":"crossref","first-page":"5371","DOI":"10.1109\/ACCESS.2020.3048319","article-title":"Tight arms race: overview of current malware threats and trends in their detection","volume":"9","author":"Caviglione","year":"2021","journal-title":"IEEE Access"},{"key":"2024121909133810800_bib19","doi-asserted-by":"crossref","DOI":"10.6028\/NIST.SP.800-53r5","author":"Joint Task Force","year":"2020","journal-title":"Security and privacy controls for information systems and organizations"},{"key":"2024121909133810800_bib20","doi-asserted-by":"crossref","first-page":"102526","DOI":"10.1016\/j.jnca.2019.102526","article-title":"The rise of machine learning for detection and classification of malware: research developments, trends and challenges","volume":"153","author":"Gibert","year":"2020","journal-title":"J Netw Comput Appl"},{"key":"2024121909133810800_bib21","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.istr.2009.03.003","article-title":"Detection of malicious code by applying machine learning classifiers on static features: a state-of-the-art survey","volume":"14","author":"Shabtai","year":"2009","journal-title":"Inf Secur Tech Rep"},{"key":"2024121909133810800_bib22","article-title":"Position paper: proposing ambient visualization and pre-attentive processing for threat detection","volume-title":"Proceedings of the 6th International Workshop on Socio-Technical Aspects in Security and Trust (STAST\u201916)","author":"Fugate","year":"2016"},{"key":"2024121909133810800_bib23","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1177\/0963721411421922","article-title":"Deliberate practice: necessary but not sufficient","volume":"20","author":"Campitelli","year":"2011","journal-title":"Curr Dir Psychol Sci"},{"key":"2024121909133810800_bib24","doi-asserted-by":"crossref","DOI":"10.1016\/B978-0-12-170150-5.50011-1","article-title":"The mind's eye in chess","volume-title":"Visual Information Processing","author":"Chase","year":"1973"},{"key":"2024121909133810800_bib25","doi-asserted-by":"crossref","DOI":"10.4135\/9781446294703.n23","article-title":"Experts\u2019 superior memory: from accumulation of chunks to building memory skills that mediate improved performance and learning","volume-title":"The SAGE Handbook of Applied Memory","author":"Ericsson","year":"2014"},{"key":"2024121909133810800_bib26","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1080\/135467897394329","article-title":"Developing expertise in decision making","volume":"3","author":"Klein","year":"1997","journal-title":"Think Reason"},{"key":"2024121909133810800_bib27","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1207\/s15327876mp0302_3","article-title":"Training situational awareness through pattern recognition in battle environments","volume":"3","author":"Kass","year":"1991","journal-title":"Mil Psychol"},{"key":"2024121909133810800_bib28","doi-asserted-by":"crossref","DOI":"10.21236\/ADA199492","author":"Klein","year":"1988","journal-title":"Rapid Decision Making on the Fire Ground"},{"key":"2024121909133810800_bib29","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1117\/12.396203","article-title":"Mine detection training based on expert skill","author":"Staszewski","year":"2000","journal-title":"Proceedings of SPIE 4038, Detection and Remediation Technologies for Mines and Minelike Targets V"},{"key":"2024121909133810800_bib30","first-page":"138","article-title":"A recognition-primed decision (RPD) model of rapid decision making","volume-title":"Decision Making in Action: Models and Methods","author":"Klein","year":"1993"},{"key":"2024121909133810800_bib31","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-540-78243-8_2","article-title":"The real work of computer network defense analysts: the analysis roles and processes that transform network data into security situation awareness","volume-title":"Proceedings of the Workshop on Visualization for Computer Security","author":"D\u2019Amico","year":"2008"},{"key":"2024121909133810800_bib32","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1177\/154193120504900304","article-title":"Achieving cyber defense situational awareness: a cognitive task analysis of information assurance analysts","volume":"49","author":"D\u2019Amico","year":"2005","journal-title":"Proc Hum Factors Ergon Soc Annu Meet"},{"key":"2024121909133810800_bib33","article-title":"Portable document format (pdf) security analysis and malware threats","author":"Blonce","year":"2008","journal-title":"Proceedings of the Black Hat Europe 2008 Conference"},{"key":"2024121909133810800_bib34","first-page":"1","volume-title":"PDF Reference","author":"Adobe Systems Incorporated","year":"2006","edition":"6th edn"},{"key":"2024121909133810800_bib35","doi-asserted-by":"crossref","first-page":"137","DOI":"10.3758\/BF03207704","article-title":"Calculation of signal detection theory measures","volume":"31","author":"Stanislaw","year":"1999","journal-title":"Behav Res Methods Instr Comput"},{"key":"2024121909133810800_bib36","doi-asserted-by":"crossref","first-page":"46","DOI":"10.3758\/BF03203619","article-title":"Corrections for extreme proportions and their biasing effects on estimated values of d'","volume":"27","author":"Hautus","year":"1995","journal-title":"Behav Res Methods Instr Comput"},{"key":"2024121909133810800_bib37","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/8369.001.0001","volume-title":"Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making","author":"Klein","year":"2009"},{"key":"2024121909133810800_bib38","doi-asserted-by":"crossref","first-page":"446","DOI":"10.1177\/0018720809344720","article-title":"False alerts in air traffic control conflict alerting system: is there a \u201ccry wolf\u201d effect?","volume":"51","author":"Wickens","year":"2009","journal-title":"Hum Factors"},{"key":"2024121909133810800_bib39","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1109\/3468.844354","article-title":"A model for types and levels of human interaction with automation","volume":"30","author":"Parasuraman","year":"2000","journal-title":"IEEE Trans Syst Man Cybern Part A Syst Humans"}],"container-title":["Journal of Cybersecurity"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/cybersecurity\/article-pdf\/10\/1\/tyae016\/61182317\/tyae016.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/cybersecurity\/article-pdf\/10\/1\/tyae016\/61182317\/tyae016.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,19]],"date-time":"2024-12-19T09:14:09Z","timestamp":1734599649000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/cybersecurity\/article\/doi\/10.1093\/cybsec\/tyae016\/7744929"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":39,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1,2]]}},"URL":"https:\/\/doi.org\/10.1093\/cybsec\/tyae016","relation":{},"ISSN":["2057-2085","2057-2093"],"issn-type":[{"type":"print","value":"2057-2085"},{"type":"electronic","value":"2057-2093"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]},"article-number":"tyae016"}}