{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T16:52:54Z","timestamp":1781283174543,"version":"3.54.1"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2021,6,28]],"date-time":"2021-06-28T00:00:00Z","timestamp":1624838400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100011039","name":"Intelligence Advanced Research Projects Activity","doi-asserted-by":"crossref","award":["2016-16031400006"],"award-info":[{"award-number":["2016-16031400006"]}],"id":[{"id":"10.13039\/100011039","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Trans. Soc. Comput."],"published-print":{"date-parts":[[2021,6,30]]},"abstract":"<jats:p>This article reports on a simulated phishing experiment targeting 6,938 faculty and staff at George Mason University. The three-week phishing campaign employed three types of phishing exploits and examined demographic, linked workstation\/network monitoring audit data, and a variety of behavioral and psychological factors measured via pre- and post-campaign surveys. While earlier research studies have reported disparate effects of gender and age, the present results suggest that these effects are not significant or are of limited strength and that other underlying factors may be more important. Specifically, significant differences in phishing susceptibility were obtained for different email contexts and based on whether individuals have been successfully phished before (these people were more likely to succumb to subsequent phishing emails in our study). Further, participants who responded to phishing exploits scored higher on impulsivity than the non-clickers. Also, participants whose survey responses indicated that they had more appropriate online \u201csecurity hygiene habits,\u201d such as checking the legitimacy of links, were less likely to be successfully phished in our campaign. Participants whose post-campaign survey responses indicated that they were suspicious of a phishing email message in our campaign were far less likely to click on the phishing link than those who were not suspicious. Similar results were obtained for judgments of pertinence of the email. Participants who indicated that they thought about the negative consequences of clicking the link were less likely to do so than participants who did not think about the negative consequences. Implications for effective training and awareness are discussed.<\/jats:p>","DOI":"10.1145\/3461672","type":"journal-article","created":{"date-parts":[[2021,6,28]],"date-time":"2021-06-28T10:19:03Z","timestamp":1624875543000},"page":"1-48","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":42,"title":["Experimental Investigation of Technical and Human Factors Related to Phishing Susceptibility"],"prefix":"10.1145","volume":"4","author":[{"given":"Frank L.","family":"Greitzer","sequence":"first","affiliation":[{"name":"PsyberAnalytix, Richland, WA USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wanru","family":"Li","sequence":"additional","affiliation":[{"name":"George Mason University, Fairfax, VA USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kathryn B.","family":"Laskey","sequence":"additional","affiliation":[{"name":"George Mason University, Fairfax, VA USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James","family":"Lee","sequence":"additional","affiliation":[{"name":"George Mason University, Fairfax, VA USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Justin","family":"Purl","sequence":"additional","affiliation":[{"name":"Human Resources Research Organization"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,6,28]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/SPW.2014.39"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2014.256"},{"key":"e_1_2_1_3_1","volume-title":"Phishing Trends and Intelligence Report. Retrieved","year":"2019","unstructured":"PhishLabs. 2019. Phishing Trends and Intelligence Report. Retrieved September 2019 from https:\/\/info.phishlabs.com\/hubfs\/2019%20PTI%20Report\/2019%20Phishing%20Trends%20and%20Intelligence%20Report.pdf. PhishLabs. 2019. Phishing Trends and Intelligence Report. Retrieved September 2019 from https:\/\/info.phishlabs.com\/hubfs\/2019%20PTI%20Report\/2019%20Phishing%20Trends%20and%20Intelligence%20Report.pdf."},{"key":"e_1_2_1_4_1","volume-title":"Internet Crime Report","author":"Federal Bureau of Investigation's Internet Crime Complaint Center (IC3). 2018.","year":"2021","unstructured":"Federal Bureau of Investigation's Internet Crime Complaint Center (IC3). 2018. Internet Crime Report . Federal Bureau of Investigation . Retrieved May 18, 2021 from https:\/\/pdf.ic3.gov\/2018_IC3Report.pdf. Federal Bureau of Investigation's Internet Crime Complaint Center (IC3). 2018. Internet Crime Report. Federal Bureau of Investigation. Retrieved May 18, 2021 from https:\/\/pdf.ic3.gov\/2018_IC3Report.pdf."},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the 53rd Hawaii International Conference on System Sciences. IEEE, 2240\u20132249","author":"Li Wanru","unstructured":"Wanru Li , James Lee , Justin Purl , Frank L. Greitzer , Bahram Yousefi , and Kathryn B. Laskey . 2020. Experimental investigation of demographic factors related to phishing . In Proceedings of the 53rd Hawaii International Conference on System Sciences. IEEE, 2240\u20132249 . https:\/\/hdl.handle.net\/10125\/64015. Wanru Li, James Lee, Justin Purl, Frank L. Greitzer, Bahram Yousefi, and Kathryn B. Laskey. 2020. Experimental investigation of demographic factors related to phishing. In Proceedings of the 53rd Hawaii International Conference on System Sciences. IEEE, 2240\u20132249. https:\/\/hdl.handle.net\/10125\/64015."},{"key":"e_1_2_1_6_1","volume-title":"Proceedings, Southwest Decision Sciences Institute Conference (SWDSI'09)","author":"Parrish James L.","year":"2009","unstructured":"James L. Parrish Jr , Janet L. Bailey , and James F. Courtney . 2009. A personality based model for determining susceptibility to phishing attacks . In Proceedings, Southwest Decision Sciences Institute Conference (SWDSI'09) . Southwest Decision Sciences Institute, 285--296. http:\/\/swdsi.org\/swdsi 2009 \/Papers\/9J05.pdf. James L. Parrish Jr, Janet L. Bailey, and James F. Courtney. 2009. A personality based model for determining susceptibility to phishing attacks. In Proceedings, Southwest Decision Sciences Institute Conference (SWDSI'09). Southwest Decision Sciences Institute, 285--296. http:\/\/swdsi.org\/swdsi2009\/Papers\/9J05.pdf."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1290958.1290968"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753383"},{"key":"e_1_2_1_9_1","volume-title":"Personality Traits and Facebook","author":"Halevi Tzipora","year":"2013","unstructured":"Tzipora Halevi , James Lewis , and Nasir Memon . 2013. Phishing , Personality Traits and Facebook . Cornell University Library . 2013 . http:\/\/arxiv.org\/abs\/1301.7643. Tzipora Halevi, James Lewis, and Nasir Memon. 2013. Phishing, Personality Traits and Facebook. Cornell University Library. 2013. http:\/\/arxiv.org\/abs\/1301.7643."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979459"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/INNOVATIONS.2012.6207742"},{"key":"e_1_2_1_12_1","first-page":"53","article-title":"Phishing in an Academic Community","volume":"44","author":"Diaz Alejandra","year":"2018","unstructured":"Alejandra Diaz , Alan T. Sherman , and Anupam Joshi . 2018 . Phishing in an Academic Community : A Study of User Susceptibility and Behavior. Cryptologia 44 , 1 (2018), 53 -- 67 . DOI:10.1080\/01611194.2019.1623343 10.1080\/01611194.2019.1623343 Alejandra Diaz, Alan T. Sherman, and Anupam Joshi. 2018. Phishing in an Academic Community: A Study of User Susceptibility and Behavior. Cryptologia 44, 1 (2018), 53--67. DOI:10.1080\/01611194.2019.1623343","journal-title":"A Study of User Susceptibility and Behavior. Cryptologia"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3336141"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1143120.1143131"},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","unstructured":"Matthew Canham Clay Posey Delainey Strickland and Michael Constantino. 2021. Phishing for long tails: Examining organizational repeat clickers and protective stewards. SAGE Open 1\u201311. https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/2158244021990656.  Matthew Canham Clay Posey Delainey Strickland and Michael Constantino. 2021. Phishing for long tails: Examining organizational repeat clickers and protective stewards. SAGE Open 1\u201311. https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/2158244021990656.","DOI":"10.1177\/2158244021990656"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.ps.41.020190.002221"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings, 23rd Australasian Conference on Information Systems (ACIS'12)","author":"Alseadoon Ibrahim","year":"2012","unstructured":"Ibrahim Alseadoon , Taizan Chan , Ernest Foo , and Juan G. Nieto . 2012. Who is more susceptible to phishing emails? A Saudi Arabian study . In Proceedings, 23rd Australasian Conference on Information Systems (ACIS'12) , Geelong, December 3--5 , 2012 . https:\/\/aisel.aisnet.org\/acis2012\/21. Ibrahim Alseadoon, Taizan Chan, Ernest Foo, and Juan G. Nieto. 2012. Who is more susceptible to phishing emails? A Saudi Arabian study. In Proceedings, 23rd Australasian Conference on Information Systems (ACIS'12), Geelong, December 3--5, 2012. https:\/\/aisel.aisnet.org\/acis2012\/21."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/1345501.1345514"},{"key":"e_1_2_1_19_1","volume-title":"7th Australian Information Warfare and Security Conference, Perth Western Australia, December 4\u20135","author":"Karakasiliotis A.","year":"2006","unstructured":"A. Karakasiliotis , S. M. Furnell , and M. Papadaki . 2006. Assessing end-user awareness of social engineering and phishing . In 7th Australian Information Warfare and Security Conference, Perth Western Australia, December 4\u20135 , 2006 . https:\/\/ro.ecu.edu.au\/isw\/12 DOI: https:\/\/doi.org\/10.4225\/75\/57a80e47aa0cb 10.4225\/75 A. Karakasiliotis, S. M. Furnell, and M. Papadaki. 2006. Assessing end-user awareness of social engineering and phishing. In 7th Australian Information Warfare and Security Conference, Perth Western Australia, December 4\u20135, 2006. https:\/\/ro.ecu.edu.au\/isw\/12 DOI: https:\/\/doi.org\/10.4225\/75\/57a80e47aa0cb"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1361-3723(07)70035-0"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2011.03.002"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.14722\/usec.2018.23016"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-70278-0_39"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025831"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.2307\/249688"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.4018\/joeuc.2012100104"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/2331465"},{"key":"e_1_2_1_28_1","volume-title":"17th Annual Conference on Systems Engineering Research (CSER)","author":"Greitzer Frank L.","year":"2019","unstructured":"Frank L. Greitzer , James D. Lee , Justin Purl , and Abbas K. Zaidi . 2019. Design and implementation of a comprehensive insider threat ontology . 17th Annual Conference on Systems Engineering Research (CSER) . Washington, DC , April 2019 . https:\/\/doi.org\/10.1016\/j.procs.2019.05.090. Published in Procedia Computer Science, 2019, 153, 36\u2013369. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S1877050919307495. 10.1016\/j.procs.2019.05.090 Frank L. Greitzer, James D. Lee, Justin Purl, and Abbas K. Zaidi. 2019. Design and implementation of a comprehensive insider threat ontology. 17th Annual Conference on Systems Engineering Research (CSER). Washington, DC, April 2019. https:\/\/doi.org\/10.1016\/j.procs.2019.05.090. Published in Procedia Computer Science, 2019, 153, 36\u2013369. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S1877050919307495."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/EMR.2019.2914612"},{"key":"e_1_2_1_30_1","volume-title":"Workshop on Usable Security","author":"Steves Michelle P.","year":"2019","unstructured":"Michelle P. Steves , Kristen K. Greene and Mary F. Theofanos. 2019. A Phish scale: Rating human phishing message detection difficulty . Workshop on Usable Security , San Diego, CA , 2019 . ISBN 1-891562-57-6. Internet Society. Michelle P. Steves, Kristen K. Greene and Mary F. Theofanos. 2019. A Phish scale: Rating human phishing message detection difficulty. Workshop on Usable Security, San Diego, CA, 2019. ISBN 1-891562-57-6. Internet Society."},{"key":"e_1_2_1_32_1","unstructured":"J. Wright. 2019. Open-Source Phishing Framework. Retrieved 2019 from https:\/\/getgophish.com\/.  J. Wright. 2019. Open-Source Phishing Framework. Retrieved 2019 from https:\/\/getgophish.com\/."},{"key":"e_1_2_1_33_1","volume-title":"McCrae","author":"Costa Paul T.","year":"1992","unstructured":"Paul T. Costa and Robert R . McCrae . 1992 . Professional Manual of the Revised NEO Personality Inventory and NEO Five-Factor Inventory . Odessa, FL : Psychological Assessment Resources . Paul T. Costa and Robert R. McCrae. 1992. Professional Manual of the Revised NEO Personality Inventory and NEO Five-Factor Inventory. Odessa, FL: Psychological Assessment Resources."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1037\/1040-3590.18.2.192"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.2307\/2136404"},{"key":"e_1_2_1_36_1","volume-title":"Quantitative Methods for Analyzing Travel Behaviour of Individuals: Some Recent Developments","author":"McFadden Daniel","year":"2021","unstructured":"Daniel McFadden . 1977. Quantitative Methods for Analyzing Travel Behaviour of Individuals: Some Recent Developments . Cowles Foundation Discussion Papers 474. Yale University : Cowles Foundation for Research in Economics. Retrieved 2021 from https:\/\/ideas.repec.org\/p\/cwl\/cwldpp\/474.html. Daniel McFadden. 1977. Quantitative Methods for Analyzing Travel Behaviour of Individuals: Some Recent Developments. Cowles Foundation Discussion Papers 474. Yale University: Cowles Foundation for Research in Economics. Retrieved 2021 from https:\/\/ideas.repec.org\/p\/cwl\/cwldpp\/474.html."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.1974.1100705"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1086\/593303"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.2168"},{"key":"e_1_2_1_40_1","volume-title":"How big is a big odds ratio? Interpreting the magnitudes of odds ratios in epidemiological studies. Communications in Statistics\u2013Simulation and Computation 39, 4","author":"Chen Henian","year":"2010","unstructured":"Henian Chen , Patricia Cohen , and Sophie Chen . 2010. How big is a big odds ratio? Interpreting the magnitudes of odds ratios in epidemiological studies. Communications in Statistics\u2013Simulation and Computation 39, 4 ( 2010 ), 860\u2013864. https:\/\/doi.org\/10.1080\/03610911003650383 10.1080\/03610911003650383 Henian Chen, Patricia Cohen, and Sophie Chen. 2010. How big is a big odds ratio? Interpreting the magnitudes of odds ratios in epidemiological studies. Communications in Statistics\u2013Simulation and Computation 39, 4 (2010), 860\u2013864. https:\/\/doi.org\/10.1080\/03610911003650383"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.4018\/IJCBPL.2015100101"},{"key":"e_1_2_1_42_1","volume-title":"Proceedings, Australasian Conference on Information Systems (ACIS'15)","author":"Butavicius Marcus","year":"2016","unstructured":"Marcus Butavicius , Kathryn Parsons , Malcolm Pattinson , and Agata McCormac . 2016 . Breaching the human firewall: Social engineering in phishing and spear-phishing emails . In Proceedings, Australasian Conference on Information Systems (ACIS'15) , Adelaide, Australia. Marcus Butavicius, Kathryn Parsons, Malcolm Pattinson, and Agata McCormac. 2016. Breaching the human firewall: Social engineering in phishing and spear-phishing emails. In Proceedings, Australasian Conference on Information Systems (ACIS'15), Adelaide, Australia."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1037\/0021-9010.73.4.688"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376570"},{"key":"e_1_2_1_45_1","volume-title":"Don't click: towards an effective anti\u2011phishing training. A comparative literature review. Human-centric Computing and Information Sciences, 10, 33","author":"Jampen Daniel","year":"2020","unstructured":"Daniel Jampen , G\u00fcrkan G\u00fcr , Thomas Sutter , and Bernhard Tellenbach . 2020. Don't click: towards an effective anti\u2011phishing training. A comparative literature review. Human-centric Computing and Information Sciences, 10, 33 ( 2020 ), 41 pages. https:\/\/doi.org\/10.1186\/s13673-020-00237-7 10.1186\/s13673-020-00237-7 Daniel Jampen, G\u00fcrkan G\u00fcr, Thomas Sutter, and Bernhard Tellenbach. 2020. Don't click: towards an effective anti\u2011phishing training. A comparative literature review. Human-centric Computing and Information Sciences, 10, 33 (2020), 41 pages. https:\/\/doi.org\/10.1186\/s13673-020-00237-7"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2013.106"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2016.101"},{"key":"e_1_2_1_48_1","volume-title":"Presentation at the DoD C-Int Social Behavioral Sciences (SBS) Summit","author":"Greitzer Frank L.","year":"2020","unstructured":"Frank L. Greitzer . 2020. \u201c Insider threats and organizational resilience: New wine in old bottles? \u201d Presentation at the DoD C-Int Social Behavioral Sciences (SBS) Summit , 2020 . The Threat Lab, Defense Personnel and Security Research Center (PERSEREC) . Frank L. Greitzer. 2020. \u201cInsider threats and organizational resilience: New wine in old bottles?\u201d Presentation at the DoD C-Int Social Behavioral Sciences (SBS) Summit, 2020. The Threat Lab, Defense Personnel and Security Research Center (PERSEREC)."}],"container-title":["ACM Transactions on Social Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461672","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3461672","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:49:05Z","timestamp":1750193345000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461672"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,28]]},"references-count":47,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,6,30]]}},"alternative-id":["10.1145\/3461672"],"URL":"https:\/\/doi.org\/10.1145\/3461672","relation":{},"ISSN":["2469-7818","2469-7826"],"issn-type":[{"value":"2469-7818","type":"print"},{"value":"2469-7826","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,28]]},"assertion":[{"value":"2020-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-06-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}