{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T13:28:54Z","timestamp":1777901334503,"version":"3.51.4"},"reference-count":28,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2023,7,3]],"date-time":"2023-07-03T00:00:00Z","timestamp":1688342400000},"content-version":"vor","delay-in-days":183,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000038","name":"NSERC","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,1,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We carry out a detailed analysis of the security advice coding method (SAcoding) of Barrera et\u00a0al., which is designed to analyze security advice in the sense of measuring actionability and categorizing advice items as practices, policies, principles, or outcomes. The main part of our analysis explores the extent to which a second coder\u2019s assignment of codes to advice items agrees with that of a first, for a dataset of 1013 security advice items nominally addressing Internet of Things devices. More broadly, we seek a deeper understanding of the soundness and utility of the SAcoding method, and the degree to which it meets the design goal of reducing subjectivity in assigning codes to security advice items. Our analysis results in suggestions for modifications to the coding tree methodology, and some recommendations. We believe the coding tree approach may be of interest for analysis of qualitative data beyond security advice datasets alone.<\/jats:p>","DOI":"10.1093\/cybsec\/tyad013","type":"journal-article","created":{"date-parts":[[2023,7,3]],"date-time":"2023-07-03T09:49:02Z","timestamp":1688377742000},"source":"Crossref","is-referenced-by-count":2,"title":["A close look at a systematic method for analyzing sets of security advice"],"prefix":"10.1093","volume":"9","author":[{"given":"David","family":"Barrera","sequence":"first","affiliation":[{"name":"School of Computer Science, Carleton University , 1125 Colonel By Drive, Ottawa, K1S 5B6 ON , Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1996-7943","authenticated-orcid":false,"given":"Christopher","family":"Bellman","sequence":"additional","affiliation":[{"name":"School of Computer Science, Carleton University , 1125 Colonel By Drive, Ottawa, K1S 5B6 ON , Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paul C","family":"van\u00a0Oorschot","sequence":"additional","affiliation":[{"name":"School of Computer Science, Carleton University , 1125 Colonel By Drive, Ottawa, K1S 5B6 ON , Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2023,7,3]]},"reference":[{"key":"2023070309485661200_bib1","doi-asserted-by":"crossref","first-page":"1362","DOI":"10.1109\/SP.2019.00013","article-title":"SoK: Security Evaluation of Home-Based IoT Deployments","volume-title":"2019 IEEE Symposium on Security and Privacy (SP), San Francisco, CA","author":"Alrawi","year":"2019"},{"key":"2023070309485661200_bib2","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1109\/MC.2017.201","article-title":"DDoS in the IoT: Mirai and Other Botnets","volume":"50","author":"Kolias","year":"2017","journal-title":"Computer"},{"key":"2023070309485661200_bib3","article-title":"CYBER; Cyber Security for Consumer Internet of Things: Baseline Requirements (ETSI EN 303 645)","author":"","year":"2020"},{"key":"2023070309485661200_bib4","article-title":"Code of Practice for Consumer IoT Security","author":"","year":"2018"},{"key":"2023070309485661200_bib5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3563392","article-title":"Security Best Practices: A Critical Analysis Using IoT as a Case Study","volume":"26","author":"Barrera","year":"2023","journal-title":"ACM Trans Priv Secur"},{"key":"2023070309485661200_bib6","article-title":"cb1013-dataset. Includes advice dataset and C1 tags","author":"Bellman","year":"2022"},{"key":"2023070309485661200_bib7","article-title":"Mapping security and privacy in the Internet of Things. Version 3 dataset","author":"Copper Horse Ltd.","year":"2019"},{"key":"2023070309485661200_bib8","doi-asserted-by":"crossref","first-page":"102989","DOI":"10.1016\/j.cose.2022.102989","article-title":"Systematic analysis and comparison of security advice datasets","volume":"124","author":"Bellman","year":"2023","journal-title":"Comput Secur"},{"key":"2023070309485661200_bib9","doi-asserted-by":"crossref","DOI":"10.4135\/9781452230153","volume-title":"Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory (third edition)","author":"Corbin","year":"2008"},{"key":"2023070309485661200_bib10","article-title":"cb1013-dataset-twocoder. Includes advice dataset and C1, C2 tags","author":"Bellman","year":"2022"},{"key":"2023070309485661200_bib11","article-title":"Mapping of IoT security recommendations, guidance and standards to the UK\u2019s code of practice for consumer IoT security","author":"UK Department for Digital, Culture, Media and Sport (DCMS)","year":"2018"},{"key":"2023070309485661200_bib12","article-title":"Artifact review and badging version 1.1","author":"Association for Computing Machinery (ACM)","year":"2020"},{"key":"2023070309485661200_bib13","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/j.cose.2016.04.001","article-title":"The Security Expertise Assessment Measure (SEAM): developing a scale for hacker expertise","volume":"60","author":"Giboney","year":"2016","journal-title":"Comp Secur"},{"key":"2023070309485661200_bib14","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1017\/CBO9780511816796.010","article-title":"Laboratory methods for assessing experts\u2019 and novices\u2019 knowledge","volume-title":"The Cambridge Handbook of Expertise and Expert Performance","author":"Chi","year":"2006"},{"key":"2023070309485661200_bib15","article-title":"Glossary Sept 20","author":"NIST Computer Security Resource Center","year":"2022"},{"key":"2023070309485661200_bib16","article-title":"RFC4949: Internet security glossary, version 2","author":"Shirey","year":"2007"},{"key":"2023070309485661200_bib17","first-page":"1","article-title":"Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice","volume":"3","author":"McDonald","year":"2019","journal-title":"Proc ACM Hum-Comput Interact"},{"key":"2023070309485661200_bib18","doi-asserted-by":"crossref","first-page":"1367","DOI":"10.1109\/SP40001.2021.00094","article-title":"They would do better if they worked together: The case of interaction problems between password managers and websites","volume-title":"2021 IEEE Symposium on Security and Privacy (SP), San Francisco, CA","author":"Huaman","year":"2021"},{"key":"2023070309485661200_bib19","first-page":"39","article-title":"\u201cMy data just goes everywhere\u201d: User mental models of the Internet and implications for privacy and security","volume-title":"SOUPS '15: Proceedings of the Eleventh USENIX Conference on Usable Privacy and Security","author":"Kang","year":"2019"},{"key":"2023070309485661200_bib20","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1145\/3133956.3134082","article-title":"Why do developers get password storage wrong? A qualitative usability study","volume-title":"CCS '17: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security","author":"Naiakshina","year":"2017"},{"key":"2023070309485661200_bib21","first-page":"1339","article-title":"\u201cI have no idea what I\u2019m doing\u201d\u2014On the usability of deploying HTTPS","volume-title":"SEC'17: Proceedings of the 26th USENIX Conference on Security Symposium","author":"Krombholz","year":"2017"},{"key":"2023070309485661200_bib22","doi-asserted-by":"crossref","first-page":"718","DOI":"10.1145\/3359789.3359800","article-title":"Will you trust this TLS certificate? Perceptions of people working in IT","volume-title":"ACSAC '19: Proceedings of the 35th Annual Computer Security Applications Conference","author":"Ukrop","year":"2019"},{"key":"2023070309485661200_bib23","first-page":"327","article-title":"\u201c...No one can hack my mind\u201d: Comparing expert and non-expert security practices","volume-title":"SOUPS '15: Proceedings of the Eleventh USENIX Conference on Usable Privacy and Security","author":"Ion","year":"2015"},{"key":"2023070309485661200_bib24","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"2023070309485661200_bib25","doi-asserted-by":"crossref","first-page":"666","DOI":"10.1145\/2976749.2978307","article-title":"How I learned to be secure: A census-representative survey of security advice sources and behavior","volume-title":"CCS '16: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security","author":"Redmiles","year":"2016"},{"key":"2023070309485661200_bib26","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1109\/SP.2016.24","article-title":"I think they\u2019re trying to tell me something: Advice sources and selection for digital security","author":"Redmiles","year":"2016","journal-title":"2016 IEEE Symposium on Security and Privacy (SP), San Jose, CA"},{"key":"2023070309485661200_bib27","article-title":"First steps toward measuring the readability of security advice","author":"Redmiles","year":"2018","journal-title":"Workshop on Technology and Consumer Protection"},{"key":"2023070309485661200_bib28","first-page":"89","article-title":"A comprehensive quality evaluation of security and privacy advice on the web","volume-title":"SEC'20: Proceedings of the 29th USENIX Conference on Security Symposium","author":"Redmiles","year":"2020"}],"container-title":["Journal of Cybersecurity"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/cybersecurity\/article-pdf\/9\/1\/tyad013\/50781802\/tyad013.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/cybersecurity\/article-pdf\/9\/1\/tyad013\/50781802\/tyad013.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,3]],"date-time":"2023-07-03T09:49:31Z","timestamp":1688377771000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/cybersecurity\/article\/doi\/10.1093\/cybsec\/tyad013\/7217003"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,1]]},"references-count":28,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,1,5]]}},"URL":"https:\/\/doi.org\/10.1093\/cybsec\/tyad013","relation":{},"ISSN":["2057-2085","2057-2093"],"issn-type":[{"value":"2057-2085","type":"print"},{"value":"2057-2093","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,1,1]]},"published":{"date-parts":[[2023,1,1]]},"article-number":"tyad013"}}