{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T11:07:19Z","timestamp":1781003239556,"version":"3.54.1"},"reference-count":24,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2019,3,12]],"date-time":"2019-03-12T00:00:00Z","timestamp":1552348800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100007299","name":"Harvard Catalyst","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007299","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health Award","doi-asserted-by":"crossref","award":["UL 1TR002541"],"award-info":[{"award-number":["UL 1TR002541"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>The study sought to understand the impact of a phishing training program on phishing click rates for employees at a single, anonymous US healthcare institution.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We stratified our population into 2 groups: offenders and nonoffenders. Offenders were defined as those that had clicked on at least 5 simulated phishing emails and nonoffenders were those that had not. We calculated click rates for offenders and nonoffenders, before and after a mandatory training program for offenders was implemented.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>A total of 5416 unique employees received all 20 campaigns during the intervention period; 772 clicked on at least 5 emails and were labeled offenders. Only 975 (17.9%) of our set clicked on 0 phishing emails over the course of the 20 campaigns; 3565 (65.3%) clicked on at least 2 emails. There was a decrease in click rates for each group over the 20 campaigns. The mandatory training program, initiated after campaign 15, did not have a substantial impact on click rates, and the offenders remained more likely to click on a phishing simulation.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>Phishing is a common threat vector against hospital employees and an important cybersecurity risk to healthcare systems. Our work suggests that, under simulation, employee click rates decrease with repeated simulation, but a mandatory training program targeted at high-risk employees did not meaningfully decrease the click rates of this population.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Employee phishing click rates decrease over time, but a mandatory training program for the highest-risk employees did not decrease click rates when compared with lower-risk employees.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocz005","type":"journal-article","created":{"date-parts":[[2019,1,21]],"date-time":"2019-01-21T20:15:36Z","timestamp":1548101736000},"page":"547-552","source":"Crossref","is-referenced-by-count":65,"title":["Evaluation of a mandatory phishing training program for high-risk employees at a US healthcare system"],"prefix":"10.1093","volume":"26","author":[{"given":"William J","family":"Gordon","sequence":"first","affiliation":[{"name":"Division of General Internal Medicine and Primary Care, Brigham and Women\u2019s Hospital, Boston, Massachusetts, USA"},{"name":"Harvard Medical School, Boston, Massachusetts, USA"},{"name":"Partners HealthCare, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adam","family":"Wright","sequence":"additional","affiliation":[{"name":"Division of General Internal Medicine and Primary Care, Brigham and Women\u2019s Hospital, Boston, Massachusetts, USA"},{"name":"Harvard Medical School, Boston, Massachusetts, USA"},{"name":"Partners HealthCare, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert J","family":"Glynn","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts, USA"},{"name":"Division of Preventive Medicine, Brigham and Women\u2019s Hospital, Boston, Massachusetts, USA"},{"name":"Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jigar","family":"Kadakia","sequence":"additional","affiliation":[{"name":"Partners HealthCare, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christina","family":"Mazzone","sequence":"additional","affiliation":[{"name":"Partners HealthCare, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elizabeth","family":"Leinbach","sequence":"additional","affiliation":[{"name":"Partners HealthCare, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adam","family":"Landman","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts, USA"},{"name":"Partners HealthCare, Boston, Massachusetts, USA"},{"name":"Department of Emergency Medicine, Brigham and Women\u2019s Hospital, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2019,3,12]]},"reference":[{"issue":"5","key":"2020110613112810700_ocz005-B1","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1056\/NEJMp1706754","article-title":"Cyberattack on Britain\u2019s National Health Service\u2014a wake-up call for modern medicine","volume":"377","author":"Clarke","year":"2017","journal-title":"N Engl J Med"},{"issue":"5","key":"2020110613112810700_ocz005-B2","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1056\/NEJMp1407326","article-title":"When \u201chacktivists\u201d target your hospital","volume":"371","author":"Nigrin","year":"2014","journal-title":"N Engl J Med"},{"issue":"8","key":"2020110613112810700_ocz005-B3","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1056\/NEJMp1707212","article-title":"Threats to information security\u2014public health implications","volume":"377","author":"Gordon","year":"2017","journal-title":"N Engl J Med"},{"issue":"6","key":"2020110613112810700_ocz005-B4","doi-asserted-by":"crossref","first-page":"878","DOI":"10.1001\/jamainternmed.2017.0336","article-title":"Hospital risk of data breaches","volume":"177","author":"Bai","year":"2017","journal-title":"JAMA Intern Med"},{"key":"2020110613112810700_ocz005-B5","year":"2016"},{"key":"2020110613112810700_ocz005-B6","year":"2016"},{"key":"2020110613112810700_ocz005-B7","author":"Spitzer"},{"key":"2020110613112810700_ocz005-B8","author":"Davis"},{"key":"2020110613112810700_ocz005-B9","year":"2018"},{"issue":"10","key":"2020110613112810700_ocz005-B10","doi-asserted-by":"crossref","first-page":"1115","DOI":"10.1007\/s11606-016-3741-z","article-title":"The big phish: cyberattacks against U.S. healthcare systems","volume":"31","author":"Wright","year":"2016","journal-title":"J Gen Intern Med"},{"issue":"6","key":"2020110613112810700_ocz005-B11","doi-asserted-by":"crossref","first-page":"584","DOI":"10.1080\/0144929X.2011.632650","article-title":"Phishing for phishing awareness","volume":"32","author":"Jansson","year":"2013","journal-title":"Behav Inf Technol"},{"key":"2020110613112810700_ocz005-B12","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1145\/1124772.1124861","volume-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems","author":"Dhamija","year":"2006"},{"key":"2020110613112810700_ocz005-B13","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1145\/1073001.1073009","volume-title":"Proceedings of the 2005 Symposium on Usable Privacy and Security","author":"Dhamija","year":"2005"},{"key":"2020110613112810700_ocz005-B14"},{"key":"2020110613112810700_ocz005-B15","first-page":"7","article-title":"Preventing phishing attacks using one time password and user machine identification","volume":"68","author":"Khan","year":"2013","journal-title":"Int J Comput Appl"},{"key":"2020110613112810700_ocz005-B16"},{"key":"2020110613112810700_ocz005-B17"},{"key":"2020110613112810700_ocz005-B18","year":"2015"},{"key":"2020110613112810700_ocz005-B19","year":"2015"},{"key":"2020110613112810700_ocz005-B20"},{"key":"2020110613112810700_ocz005-B21"},{"key":"2020110613112810700_ocz005-B22","author":"Chandramouli","year":"2016"},{"key":"2020110613112810700_ocz005-B23","author":"Duke","year":"2017"},{"key":"2020110613112810700_ocz005-B24"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/26\/6\/547\/34153669\/ocz005.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/26\/6\/547\/34153669\/ocz005.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T19:36:36Z","timestamp":1604691396000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/26\/6\/547\/5376646"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,3,12]]},"references-count":24,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2019,3,12]]},"published-print":{"date-parts":[[2019,6,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocz005","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,6]]},"published":{"date-parts":[[2019,3,12]]}}}