{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T12:02:20Z","timestamp":1777118540342,"version":"3.51.4"},"reference-count":90,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2022,5,23]],"date-time":"2022-05-23T00:00:00Z","timestamp":1653264000000},"content-version":"vor","delay-in-days":142,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,1,28]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Cybersecurity awareness (CSA) is not just about knowing, but also transforming things learned into practice. It is a continuous process that needs to be adjusted in subsequent iterations to improve its usability as well as sustainability. This is possible only if a CSA program is reviewed and evaluated timely. Review and evaluation of an awareness program offer an insight into the program's effectiveness on the audience and organization, an invaluable piece of information for the continuous improvement of the program. Further, it provides the information required by the management and sponsor to decide on whether to invest in the program or not. Despite these advantages, there does not exist a common understanding of what factors to measure and how to measure them during the evaluation process. As a result, we have proposed evaluation metrics for the purpose. In order to do so, we performed a literature review of 32 papers mainly to extract the following data: (i) what factors did the paper measure, and (ii) how did it measure the factors? Next, we adapted the European Literacy Policy Network's four indicators (i.e. impact, sustainability, accessibility, and monitoring) for awareness evaluation to make it appropriate for evaluating a CSA program. We believe that measuring all four indicators will contribute to making the evaluation process systematic, complete, and replicable. More importantly, it will help to produce more inclusive, accurate, and usable results for the future enhancement of the program.<\/jats:p>","DOI":"10.1093\/cybsec\/tyac006","type":"journal-article","created":{"date-parts":[[2022,5,23]],"date-time":"2022-05-23T10:56:35Z","timestamp":1653303395000},"source":"Crossref","is-referenced-by-count":57,"title":["Developing metrics to assess the effectiveness of cybersecurity awareness program"],"prefix":"10.1093","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7138-3411","authenticated-orcid":false,"given":"Sunil","family":"Chaudhary","sequence":"first","affiliation":[{"name":"Department of Information Security and Communication Technology, Norwegian University of Science and Technology , Teknologivegen 22, 2815 Gj\u00f8vik, Norway"}]},{"given":"Vasileios","family":"Gkioulos","sequence":"additional","affiliation":[{"name":"Department of Information Security and Communication Technology, Norwegian University of Science and Technology , Teknologivegen 22, 2815 Gj\u00f8vik, Norway"}]},{"given":"Sokratis","family":"Katsikas","sequence":"additional","affiliation":[{"name":"Department of Information Security and Communication Technology, Norwegian University of Science and Technology , Teknologivegen 22, 2815 Gj\u00f8vik, Norway"}]}],"member":"286","published-online":{"date-parts":[[2022,5,23]]},"reference":[{"key":"2022121515105139100_bib1","article-title":"More than half of personal data breaches caused by human error","author":"Williams","year":"2019"},{"key":"2022121515105139100_bib2","author":"IBM Global Technology Services","year":"2014","journal-title":"IBM Security Services 2014 Cyber Security Intelligence Index Analysis of cyber attack and incident data from IBM's worldwide security operations"},{"key":"2022121515105139100_bib3","article-title":"The human factor in IT security: How employees are making businesses vulnerable from within","author":"Kaspersky","year":"2018"},{"key":"2022121515105139100_bib4","doi-asserted-by":"crossref","article-title":"Building an information technology security awareness and training program","author":"Wilson","DOI":"10.6028\/NIST.SP.800-50"},{"key":"2022121515105139100_bib5","article-title":"The new users' guide: How to raise information security awareness","author":"ENISA","year":"2010"},{"key":"2022121515105139100_bib6","doi-asserted-by":"crossref","DOI":"10.1109\/DEXA.2014.71","article-title":"Specifying IT security awareness","volume-title":"Proceedings of the Twenty-fifth International Workshop on Database and Expert Systems Applications","author":"H\u00e4nsch","year":"2014"},{"key":"2022121515105139100_bib7","article-title":"Cyber security awareness campaigns: why do they fail to change behaviour?","author":"Bada","year":"2015","journal-title":"Proceedings of the International Conference on Cyber Security for Sustainable Society"},{"key":"2022121515105139100_bib8","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/S1386-5056(00)00112-X","article-title":"Health care management and information system security: awareness, training or education?","volume":"60","author":"Katsikas","year":"2000","journal-title":"Int J Med Inf"},{"key":"2022121515105139100_bib9","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1080\/15332861.2010.487415","article-title":"Influence of awareness and training on cyber security","volume":"9","author":"McCrohan","year":"2010","journal-title":"J Internet Commer"},{"key":"2022121515105139100_bib10","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/S1353-4858(17)30122-8","article-title":"Security education and awareness: just let them burn?","volume":"2017","author":"Furnell","year":"2017","journal-title":"Netw Secur"},{"key":"2022121515105139100_bib11","article-title":"Information security awareness initiatives: current practice and the measurement of success","author":"ENISA"},{"key":"2022121515105139100_bib12","doi-asserted-by":"crossref","article-title":"Exploring the effectiveness of transit security awareness campaigns in the San Francisco Bay area","author":"Rohlich","DOI":"10.3141\/2146-12"},{"key":"2022121515105139100_bib13","article-title":"Security awareness metrics","author":"Spitzner"},{"key":"2022121515105139100_bib14","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1093\/acprof:oso\/9780199688890.003.0003","article-title":"How can we measure awareness? An overview of current methods","volume-title":"Behavioural Methods in Consciousness Research","author":"Timmermans","year":"2015"},{"key":"2022121515105139100_bib15","first-page":"26","article-title":"A behavior model for persuasive design","author":"Fogg","year":"2009","journal-title":"Proceedings of the Fourth International Conference on Persuasive Technology"},{"key":"2022121515105139100_bib16","article-title":"CSI computer crime & security survey","author":"Richardson"},{"key":"2022121515105139100_bib17","volume-title":"Security Awareness Training: It's Not Just For Compliance","author":"Monaha","year":"2014"},{"key":"2022121515105139100_bib18","first-page":"15","article-title":"Current issues of metrics for information security awareness","author":"Fertig","year":"2020","journal-title":"Proceedings of the Twenty-Eighth European Conference on Information Systems"},{"key":"2022121515105139100_bib19","article-title":"Security awareness report","author":"Spitzner","year":"2019"},{"key":"2022121515105139100_bib20","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1080\/00405841.2016.1148989","article-title":"Formatibe and summative assessment in the classroom","volume":"55","author":"Dixson","year":"2016","journal-title":"Theory Into Practice"},{"key":"2022121515105139100_bib21","first-page":"258","article-title":"DSAPE: dynamic security awareness program evaluation","author":"Manifavas","year":"2014","journal-title":"Proceedings of the Sixteenth International Conference on Human-Computer Interaction"},{"key":"2022121515105139100_bib22","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1108\/ICS-07-2018-0080","article-title":"Developing cybersecurity education and awareness programmers for small and medium-sized enterprises (SMEs)","volume":"27","author":"Bada","year":"2019","journal-title":"Inf Comput Secur"},{"key":"2022121515105139100_bib23","doi-asserted-by":"crossref","DOI":"10.1007\/11962977_10","article-title":"Can an early warning system for home users and SMEs make a difference? A field study","volume-title":"Proceedings of the International Workshop on Critical Information Infrastructures Security","author":"Gattiker","year":"2006"},{"key":"2022121515105139100_bib24","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.compedu.2008.06.011","article-title":"The impact of information richness on information security awareness training effectiveness","volume":"52","author":"Shaw","year":"2009","journal-title":"Comput Edu"},{"key":"2022121515105139100_bib25","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1145\/3313831.3376385","article-title":"Evaluating the information security awareness of smartphone users","volume-title":"Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI'20)","author":"Bitton","year":"2020"},{"key":"2022121515105139100_bib26","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1016\/j.cose.2006.02.008","article-title":"A prototype for assessing information security awareness","volume":"25","author":"Kruger","year":"2006","journal-title":"Comput Secur"},{"key":"2022121515105139100_bib27","first-page":"xiii","article-title":"Analyzing the past to prepare for the future: writing a literature review","volume":"26","author":"Webster","year":"2002","journal-title":"MIS Quart"},{"key":"2022121515105139100_bib28","author":"Google Scholar"},{"key":"2022121515105139100_bib29","author":"Microsoft Academic"},{"key":"2022121515105139100_bib30","volume-title":"Procedures for Performing Systematic Reviews","author":"Kitchenham","year":"2004"},{"key":"2022121515105139100_bib90","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1111\/jebm.12266","article-title":"Gray literature: an important resource in systematic reviews","volume":"10","author":"Paez","year":"2017","journal-title":"J Evid Based Med"},{"key":"2022121515105139100_bib31","article-title":"D9.13 Awareness effectiveness study","author":"Chaudhary","year":"2021"},{"key":"2022121515105139100_bib32","article-title":"D9.11: SME cybersecurity awareness program 2","author":"Chaudhary"},{"key":"2022121515105139100_bib33","first-page":"22","article-title":"Using phishing for user email security awareness","author":"Dodge","year":"2006","journal-title":"Proceedings of the IFIP TC-11 Twenty-First International Information Security Conference (SEC 2006)"},{"key":"2022121515105139100_bib34","article-title":"A framework for evaluating ICT security awareness","author":"Kruger","year":"2006","journal-title":"Proceedings of the ISSA 2006 from Insight to Foresight Conference, July 5-7"},{"key":"2022121515105139100_bib35","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1108\/09685221011095236","article-title":"A vocabulary test to assess information security awareness","volume":"18","author":"Kruger","year":"2010","journal-title":"Inf Manag Comput Secur"},{"key":"2022121515105139100_bib36","doi-asserted-by":"crossref","first-page":"432","DOI":"10.1016\/j.cose.2009.12.005","article-title":"Improving information security awareness and behavior through dialogue, participation, and collective reflection: an intervention study","volume":"29","author":"Albrechtsen","year":"2010","journal-title":"Comput Secur"},{"key":"2022121515105139100_bib37","first-page":"10862","article-title":"Effectiveness of information security awareness methods based on psychological theories","volume":"5","author":"Khan","year":"2011","journal-title":"Afr J Bus Manag"},{"key":"2022121515105139100_bib38","first-page":"9","article-title":"Measuring an information security awareness program","volume":"15","author":"Wolf","year":"2011","journal-title":"Rev Bus Inf Syst"},{"key":"2022121515105139100_bib39","first-page":"5","article-title":"Information security awareness in university: maintaining learnability, performance, and adaptability through roles of responsibility","author":"Ahlan","year":"2011","journal-title":"Proceedings of the Seventh International Conference on Information Assurance and Security (IAS)"},{"key":"2022121515105139100_bib40","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1108\/09593841211254358","article-title":"Analyzing trajectories of information security awareness","volume":"25","author":"Tsohou","year":"2012","journal-title":"Inf Technol People"},{"key":"2022121515105139100_bib41","article-title":"End user information security awareness programs for improving information security in banking organizations: preliminary results from an exploratory study","volume-title":"Proceedings of the AIS SIGSEC Workshop on Information Security & Privacy (WISP2013)","author":"Bauer","year":"2013"},{"key":"2022121515105139100_bib42","first-page":"69","article-title":"Ignorance to awareness: towards an information security awareness process","volume":"104","author":"Gundu","year":"2013","journal-title":"South Afr Inst Elect Eng"},{"key":"2022121515105139100_bib43","first-page":"26","article-title":"Development of user's information security awareness questionnaire (UISAQ)","volume-title":"Proceedings of the International Convention MIPRO","author":"Velki","year":"2014"},{"key":"2022121515105139100_bib44","first-page":"62","article-title":"The perceived effectiveness of information security awareness","volume":"6","author":"Prah","year":"2016","journal-title":"Inf Knowl Manag"},{"key":"2022121515105139100_bib45","first-page":"58","article-title":"Blind spot: do you know the effectiveness of your information security awareness raising program?","volume":"15","author":"Scholl","year":"2017","journal-title":"Syst Cybernet Inf"},{"key":"2022121515105139100_bib46","first-page":"11","article-title":"Impact of security awareness training on phishing click-through rates","author":"Carella","year":"2017","journal-title":"Proceedings of the IEEE International Conference on Big Data"},{"key":"2022121515105139100_bib47","first-page":"25","article-title":"Information security awareness level measurement for employee: case study at Ministry of Research, Technology, and Higher Education","author":"Wahyudiwan","year":"2017","journal-title":"Proceedings of the Third International Conference on Science in Information Technology"},{"key":"2022121515105139100_bib48","first-page":"8","article-title":"Effectiveness of cyber security awareness program for young children: a case study in UAE","volume":"3","author":"Shamsi","year":"2019","journal-title":"Int J Inf Technol Lang Stud"},{"key":"2022121515105139100_bib49","article-title":"Deliver security awareness training, then repeat: {Deliver, Measure Efficacy}","author":"Gundu","year":"2019","journal-title":"Prroceedings of the Conference on Information Communications Technology and Society (ICTAS)"},{"key":"2022121515105139100_bib50","doi-asserted-by":"crossref","first-page":"1276","DOI":"10.1108\/ITP-06-2018-0278","article-title":"Online social network security awareness: mass interpersonal persuasion using a Facebook app","volume":"32","author":"Ikhalia","year":"2019","journal-title":"Inf Technol People"},{"key":"2022121515105139100_bib51","doi-asserted-by":"crossref","first-page":"e02010","DOI":"10.1016\/j.heliyon.2019.e02010","article-title":"Effectiveness of and user preferences for security awareness training methodologies","volume":"5","author":"Tschakert","year":"2019","journal-title":"Heliyon"},{"key":"2022121515105139100_bib52","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1109\/MC.2020.3001959","article-title":"Security awareness training for the workforce: moving beyond \u201ccheck-the-box\u201d compliance","volume":"53","author":"Haney","year":"2020","journal-title":"Computer"},{"key":"2022121515105139100_bib53","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1108\/IMCS-10-2013-0078","article-title":"A study of information security awareness in Australian government organisations","volume":"22","author":"Parsons","year":"2014","journal-title":"Inf Manag Comput Secur"},{"key":"2022121515105139100_bib54","article-title":"Examining the effects of knowledge, attitude and behaviour on information security awareness: A case on SME","author":"Kaur","year":"2013","journal-title":"Proceedings of the International Conference on Research and Innovation in Information Systems (ICRIIS)"},{"key":"2022121515105139100_bib55","article-title":"The effectiveness of online gaming as part of a security awareness program","author":"Labuschagne","year":"2014","journal-title":"Proceedings of the Thirteenth European Conference on Cyber Warfare and Security"},{"key":"2022121515105139100_bib56","first-page":"1","article-title":"Security awareness level of smartphone users: an exploratory case study","volume":"2019","author":"Koyuncu","year":"2019","journal-title":"Mob Inf Syst"},{"key":"2022121515105139100_bib57","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1016\/j.procs.2015.12.151","article-title":"Information security awareness at the knowledge-based institution: its antecedents and measures","volume":"72","author":"Ahlan","year":"2015","journal-title":"Proc Comput Sci"},{"key":"2022121515105139100_bib58","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1108\/09685220810908787","article-title":"A cross-cultural investigation of situational information security awareness programs","volume":"16","author":"Chen","year":"2008","journal-title":"Inf Manag Comput Secur"},{"key":"2022121515105139100_bib59","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1016\/j.istr.2010.05.002","article-title":"The positive outcomes of information security awareness training in companies: a case study","volume":"14","author":"Emina\u011fao\u011flu","year":"2009","journal-title":"Inf Secur Tech Rep"},{"key":"2022121515105139100_bib60","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1080\/19393555.2012.747234","article-title":"How effective is your security awareness program? An evaluation methodology","volume":"21","author":"Rantos","year":"2012","journal-title":"Inf Secur J Glob Perspect"},{"key":"2022121515105139100_bib61","doi-asserted-by":"crossref","DOI":"10.1109\/ARES.2010.27","article-title":"An analysis of information security awareness within home and work environments","author":"Talib","year":"2010","journal-title":"Proceedings of the International Conference on Availability, Reliability and Security"},{"key":"2022121515105139100_bib62","doi-asserted-by":"crossref","first-page":"523","DOI":"10.2307\/25750690","article-title":"Information security policy compliance: an empirical study of rationality-based beliefs and information security awareness","volume":"34","author":"Bulgurcu","year":"2010","journal-title":"MIS Quart"},{"key":"2022121515105139100_bib63","article-title":"Assessing the effectiveness of security awareness training","author":"Kruse"},{"key":"2022121515105139100_bib64","article-title":"Awareness is only the first step: a framework for progressive engagement of staff in cyber security","author":"Beyer"},{"key":"2022121515105139100_bib65","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1016\/j.cose.2013.12.003","article-title":"Determining employee awareness using the Human Aspects of Information Security Questionnaire (HAIS-Q)","volume":"42","author":"Parsons","year":"2014","journal-title":"Comput Secur"},{"key":"2022121515105139100_bib66","article-title":"Scaling the security wall: developing a security behaviour intention scale (SeBIS)","author":"Egelman","year":"2015","journal-title":"Proceedings of the Thirty-Third Annual ACM Conference on Human Factors in Computing Systems"},{"key":"2022121515105139100_bib67","article-title":"A self-report measure of end-user security attitudes (SA-6)","author":"Faklaris","year":"2019","journal-title":"Proceedings of the USENIX Symposium on Usable Privacy and Security (SOUPS)"},{"key":"2022121515105139100_bib68","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1108\/ICS-04-2017-0020","article-title":"Factors in an end user security expertise instrument","volume":"25","author":"Rajivan","year":"2017","journal-title":"Inf Comput Secur"},{"key":"2022121515105139100_bib69","doi-asserted-by":"crossref","first-page":"e00346","DOI":"10.1016\/j.heliyon.2017.e00346","article-title":"Human factors in cybersecurity, examining the link between Internet addiction, impulsivity, attitudes towards cybersecurity, and risky cybersecurity behaviours","volume":"3","author":"Hadlington","year":"2017","journal-title":"Heliyon"},{"key":"2022121515105139100_bib70","first-page":"1","article-title":"An abbreviated impulsiveness scale (ABIS) constructed through confirmatory factor analysis of the BIS-11","volume":"2","author":"Coutlee","year":"2014","journal-title":"Arch Sci Psychol"},{"key":"2022121515105139100_bib71","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1089\/109493102760275581","article-title":"Validation of a new scale for measuring problematic internet use: implications for pre-employment screening","volume":"5","author":"Davis","year":"2002","journal-title":"Cyberpsychol Behav"},{"key":"2022121515105139100_bib72","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1016\/j.cose.2015.10.002","article-title":"Analysis of personal information security behavior and awareness","volume":"56","author":"\u00d6g\u02d8\u00fct\u00e7\u00fc","year":"2016","journal-title":"Comput Secur"},{"key":"2022121515105139100_bib73","article-title":"Smartphone security behavioral scale: a new psychometric measurement for smartphone security","author":"Huang"},{"key":"2022121515105139100_bib74","doi-asserted-by":"crossref","DOI":"10.1109\/MIPRO.2014.6859789","article-title":"Development of Users' Information Security Awareness Questionnaire (UISAQ)\u2014ongoing work","volume-title":"Proceedings of the Thirty-Seventh International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO)","author":"Velki","year":"2014"},{"key":"2022121515105139100_bib75","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1016\/B978-0-12-803843-7.00033-8","article-title":"Security education, training, and awareness","volume-title":"Computer and Information Security Handbook","author":"Caballero","year":"2017"},{"key":"2022121515105139100_bib76","article-title":"Security awareness for senior management","author":"Spitzner"},{"key":"2022121515105139100_bib77","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1201\/1086\/45241.14.2.20050501\/88292.6","article-title":"Implementing an information security awareness program","volume":"14","author":"Peltier","year":"2005","journal-title":"Inf Syst Secur"},{"key":"2022121515105139100_bib78","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1080\/0144929X.2012.708787","article-title":"User preference of cyber security awareness delivery methods","volume":"33","author":"Abawajy","year":"2014","journal-title":"Behav Inf Technol"},{"key":"2022121515105139100_bib79","first-page":"45","article-title":"How to increase cybersecurity awareness","volume":"2","author":"Nachin","year":"2019","journal-title":"ISACA J"},{"key":"2022121515105139100_bib80","first-page":"293","article-title":"Integrating the principles of DGBL, CSCL and playability in the design of social videogames: a case of study","volume-title":"Student Usability in Educational Software and Games: Improving Experiences","author":"Gonz\u00e1lez","year":"2012"},{"key":"2022121515105139100_bib81","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-51971-1_22","article-title":"Schoolteacher preference of cyber-safety awareness delivery methods: a South African study","volume-title":"Artificial Intelligence and Bioinspired Computational Methods. CSOC 2020. Advances in Intelligent Systems and Computing","author":"Mabitle","year":"2020"},{"key":"2022121515105139100_bib82","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1037\/0022-0663.94.3.545","article-title":"Interest, learning, and the psychological processes that mediate their relationship","volume":"94","author":"Ainley","year":"2002","journal-title":"J Educ Psychol"},{"key":"2022121515105139100_bib83","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1177\/0963721414522812","article-title":"The nature and power of interests","volume":"23","author":"Rounds","year":"2014","journal-title":"Curr Dir Psychol Sci"},{"key":"2022121515105139100_bib84","article-title":"Intrusive and non-intrusive evaluation of ambient displays","volume-title":"Proceedings of the First International Workshop on Ambient Information Systems, Collocated at Pervasive","author":"Shen","year":"2007"},{"key":"2022121515105139100_bib85","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-66504-3_19","article-title":"Analysing simulated phishing campaigns for staff","volume-title":"Proceedings of the ESORICS Second Workshop on Security, Privacy, Organizations, and Systems","author":"Volkamer","year":"2020"},{"key":"2022121515105139100_bib86","doi-asserted-by":"crossref","DOI":"10.1145\/3368308.3415443","article-title":"Clickthrough testing for real-world phishing simulations","author":"Dupuis","year":"2020","journal-title":"Proceedings of the Twenty-First Annual Conference on Information Technology Education"},{"key":"2022121515105139100_bib87","article-title":"Define S.M.A.R.T IT Security Goals. (ISC)2","author":"Mustaca","year":"2013"},{"key":"2022121515105139100_bib88","volume-title":"Indicators for Evaluation of Awareness and Fundraising for Low Literacy in Europe","author":"Ceneric","year":"2014"},{"key":"2022121515105139100_bib89","article-title":"Define ethical and quality evaluation standards","author":"BetterEvaluation"}],"container-title":["Journal of Cybersecurity"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/cybersecurity\/article-pdf\/8\/1\/tyac006\/47918864\/tyac006.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/cybersecurity\/article-pdf\/8\/1\/tyac006\/47918864\/tyac006.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,15]],"date-time":"2022-12-15T23:09:55Z","timestamp":1671145795000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/cybersecurity\/article\/doi\/10.1093\/cybsec\/tyac006\/6590603"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,1]]},"references-count":90,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1,28]]}},"URL":"https:\/\/doi.org\/10.1093\/cybsec\/tyac006","relation":{},"ISSN":["2057-2085","2057-2093"],"issn-type":[{"value":"2057-2085","type":"print"},{"value":"2057-2093","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,1,1]]},"published":{"date-parts":[[2022,1,1]]},"article-number":"tyac006"}}