{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T12:54:33Z","timestamp":1780491273730,"version":"3.54.1"},"reference-count":46,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2020,1,1]],"date-time":"2020-01-01T00:00:00Z","timestamp":1577836800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,1,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>As organizations continue to invest in phishing awareness training programs, many chief information security officers (CISOs) are concerned when their training exercise click rates are high or variable, as they must justify training budgets to organization officials who question the efficacy of awareness training when click rates are not declining. We argue that click rates should be expected to vary based on the difficulty of the phishing email for a target audience. Past research has shown that when the premise of a phishing email aligns with a user\u2019s work context, it is much more challenging for users to detect a phish. Given this, we propose a Phish Scale, so CISOs and phishing training implementers can easily rate the difficulty of their phishing exercises and help explain associated click rates. We base our scale on past research in phishing cues and user context, and apply the scale to previously published and new data from enterprise-based phishing exercises. The Phish Scale performed well with the current phishing dataset, but future work is needed to validate it with a larger variety of phishing emails. The Phish Scale shows great promise as a tool to help frame data sharing on phishing exercise click rates across sectors.<\/jats:p>","DOI":"10.1093\/cybsec\/tyaa009","type":"journal-article","created":{"date-parts":[[2020,9,14]],"date-time":"2020-09-14T16:24:11Z","timestamp":1600100651000},"source":"Crossref","is-referenced-by-count":35,"title":["Categorizing human phishing difficulty: a Phish Scale"],"prefix":"10.1093","volume":"6","author":[{"given":"Michelle","family":"Steves","sequence":"first","affiliation":[{"name":"National Institute of Standards and Technology Gaithersberg, MD 20899, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kristen","family":"Greene","sequence":"additional","affiliation":[{"name":"National Institute of Standards and Technology Gaithersberg, MD 20899, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mary","family":"Theofanos","sequence":"additional","affiliation":[{"name":"National Institute of Standards and Technology Gaithersberg, MD 20899, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2020,9,14]]},"reference":[{"key":"2020091412240618100_tyaa009-B1","year":"2019"},{"key":"2020091412240618100_tyaa009-B2","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1109\/MC.2018.2701632","article-title":"No phishing beyond this point","volume":"51","author":"Greene","year":"2018","journal-title":"IEEE Comp Cybertrust Column"},{"key":"2020091412240618100_tyaa009-B3","author":"Greene","year":"2018"},{"key":"2020091412240618100_tyaa009-B4","author":"Newman","year":"2018"},{"key":"2020091412240618100_tyaa009-B5","year":"2018"},{"key":"2020091412240618100_tyaa009-B6"},{"key":"2020091412240618100_tyaa009-B7","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1177\/0018720818780472","article-title":"Hacking the human: the prevalence paradox in cybersecurity","volume":"60","author":"Sawyer","year":"2018","journal-title":"Human Factors"},{"key":"2020091412240618100_tyaa009-B8","doi-asserted-by":"crossref","first-page":"1465","DOI":"10.1126\/science.aap8731","article-title":"Prevalence-induced concept change in human judgement","volume":"360","author":"Levari","year":"2018","journal-title":"Science"},{"key":"2020091412240618100_tyaa009-B9","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1080\/00223980.1975.9915803","article-title":"A protection motivation theory of fear appeals and attitude change","volume":"91","author":"Rogers","year":"1975","journal-title":"J Psychol"},{"key":"2020091412240618100_tyaa009-B10","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1287\/isre.2016.0680","article-title":"Coping responses in phishing detection: an investigation of antecedents and consequences","volume":"28","author":"Wang","year":"2017","journal-title":"Inf Syst Res"},{"key":"2020091412240618100_tyaa009-B11","doi-asserted-by":"crossref","first-page":"576","DOI":"10.1016\/j.dss.2011.03.002","article-title":"Why do people get phished? Testing individual differences in phishing vulnerability within an integrated, information processing model","volume":"51","author":"Vishwanath","year":"2011","journal-title":"Decis Supp Syst"},{"key":"2020091412240618100_tyaa009-B12","volume-title":"Thinking, Fast and Slow","author":"Kahneman","year":"2011"},{"key":"2020091412240618100_tyaa009-B13","doi-asserted-by":"crossref","first-page":"1146","DOI":"10.1177\/0093650215627483","article-title":"Suspicion, cognition, and automaticity model of phishing susceptibility","volume":"45","author":"Vishwanath","year":"2018","journal-title":"Comm Res"},{"key":"2020091412240618100_tyaa009-B14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ijhcs.2018.06.004","article-title":"Exploring susceptibility to phishing in the workplace","volume":"120","author":"Williams","year":"2018","journal-title":"Int J Human-Comp Stud"},{"key":"2020091412240618100_tyaa009-B15","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.cose.2018.03.012","article-title":"Evaluating the applicability of the double system lens model to the analysis of phishing email judgments","volume":"77","author":"Molinaro","year":"2018","journal-title":"Comp Sec"},{"key":"2020091412240618100_tyaa009-B16","article-title":"Do users focus on the correct cues to differentiate between phishing and genuine emails?","author":"Parsons","year":"2015","journal-title":"Australasian Conference on Information Systems"},{"key":"2020091412240618100_tyaa009-B17","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1109\/MSP.2013.106","article-title":"Going spear phishing: exploring embedded training and awareness","volume":"12","author":"Caputo","year":"2014","journal-title":"IEEE Sec Priv"},{"key":"2020091412240618100_tyaa009-B18","author":"Steves","year":"2019"},{"key":"2020091412240618100_tyaa009-B19","first-page":"3469","author":"Blythe","year":"2011"},{"key":"2020091412240618100_tyaa009-B20","doi-asserted-by":"crossref","first-page":"1158","DOI":"10.1177\/0018720816665025","article-title":"Quantifying phishing susceptibility for detection and behavior decisions","volume":"58","author":"Canfield","year":"2016","journal-title":"Human Fact"},{"key":"2020091412240618100_tyaa009-B21","first-page":"79","author":"Downs","year":"2006"},{"key":"2020091412240618100_tyaa009-B22","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/S1361-3723(07)70035-0","article-title":"Phishing: can we spot the signs?","volume":"2007","author":"Furnell","year":"2007","journal-title":"Comp Fraud Sec"},{"key":"2020091412240618100_tyaa009-B23","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1023\/B:GRUP.0000021839.04093.5d","article-title":"Where did they go wrong? An analysis of the failure of knowledgeable internet consumers to detect deception over the internet","volume":"13","author":"Grazioli","year":"2004","journal-title":"Group Decis Negot"},{"key":"2020091412240618100_tyaa009-B24","doi-asserted-by":"crossref","DOI":"10.1002\/9781119183624","volume-title":"Phishing Dark Waters","author":"Hadnagy","year":"2015"},{"key":"2020091412240618100_tyaa009-B25","first-page":"1","article-title":"The human factor in phishing","volume":"7","author":"Jakobsson","year":"2007","journal-title":"Priv Sec Consum Inform"},{"key":"2020091412240618100_tyaa009-B26","author":"Jakobsson","year":"2007"},{"key":"2020091412240618100_tyaa009-B27","author":"Karakasiliotis","year":"2006"},{"key":"2020091412240618100_tyaa009-B28","first-page":"366","volume-title":"IFIP International Information Security Conference","author":"Parsons","year":"2013"},{"key":"2020091412240618100_tyaa009-B29","first-page":"345","author":"Wang","year":"2012"},{"key":"2020091412240618100_tyaa009-B30","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1007\/s10726-009-9167-9","article-title":"Where did they go right?\u2019 Understanding the deception in phishing communications","volume":"19","author":"Wright","year":"2010","journal-title":"Group Decis Negot"},{"key":"2020091412240618100_tyaa009-B31","first-page":"2079","author":"Han","year":"2016"},{"key":"2020091412240618100_tyaa009-B32","doi-asserted-by":"crossref","first-page":"835","DOI":"10.1108\/OIR-03-2012-0037","article-title":"Understanding persuasive elements in phishing e-mails: a categorical content and semantic network analysis","volume":"37","author":"Kim","year":"2013","journal-title":"Online Inform Rev"},{"key":"2020091412240618100_tyaa009-B33","first-page":"1065","author":"Egelman","year":"2008"},{"key":"2020091412240618100_tyaa009-B34","author":"Tsow","year":"2007"},{"key":"2020091412240618100_tyaa009-B35","first-page":"581","author":"Dhamija","year":"2006"},{"key":"2020091412240618100_tyaa009-B36","first-page":"722","author":"Fogg","year":"2003"},{"key":"2020091412240618100_tyaa009-B37","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.ijhcs.2015.05.005","article-title":"Why phishing still works: user strategies for combating phishing attacks","volume":"82","author":"Alsharnouby","year":"2015","journal-title":"Int J Human-Comp Stud"},{"key":"2020091412240618100_tyaa009-B38","first-page":"6452","article-title":"Transfer of learning","volume":"2","author":"Perkins","year":"1992","journal-title":"Int Encycl Edu"},{"key":"2020091412240618100_tyaa009-B39"},{"key":"2020091412240618100_tyaa009-B40","author":"Smith","year":"2015"},{"key":"2020091412240618100_tyaa009-B41","doi-asserted-by":"crossref","first-page":"229","DOI":"10.3233\/WOR-131659","article-title":"Measuring individual work performance: identifying and selecting indicators","volume":"48","author":"Koopmans","year":"2014","journal-title":"Work"},{"key":"2020091412240618100_tyaa009-B42"},{"key":"2020091412240618100_tyaa009-B43"},{"key":"2020091412240618100_tyaa009-B44","author":"Tamborello","year":"2017"},{"key":"2020091412240618100_tyaa009-B45","year":"2018"},{"key":"2020091412240618100_tyaa009-B46"}],"container-title":["Journal of Cybersecurity"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/cybersecurity\/article-pdf\/6\/1\/tyaa009\/33746006\/tyaa009.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/cybersecurity\/article-pdf\/6\/1\/tyaa009\/33746006\/tyaa009.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,9,14]],"date-time":"2020-09-14T16:24:18Z","timestamp":1600100658000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/cybersecurity\/article\/doi\/10.1093\/cybsec\/tyaa009\/5905453"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,1]]},"references-count":46,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,1,1]]}},"URL":"https:\/\/doi.org\/10.1093\/cybsec\/tyaa009","relation":{},"ISSN":["2057-2085","2057-2093"],"issn-type":[{"value":"2057-2085","type":"print"},{"value":"2057-2093","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020]]},"published":{"date-parts":[[2020,1,1]]},"article-number":"tyaa009"}}