{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,9]],"date-time":"2025-12-09T18:10:29Z","timestamp":1765303829353,"version":"3.41.0"},"reference-count":86,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2021,10,29]],"date-time":"2021-10-29T00:00:00Z","timestamp":1635465600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Internet Technol."],"published-print":{"date-parts":[[2022,5,31]]},"abstract":"<jats:p>Computer users are generally faced with difficulties in making correct security decisions. While an increasingly fewer number of people are trying or willing to take formal security training, online sources including news, security blogs, and websites are continuously making security knowledge more accessible. Analysis of cybersecurity texts from this grey literature can provide insights into the trending topics and identify current security issues as well as how cyber attacks evolve over time. These in turn can support researchers and practitioners in predicting and preparing for these attacks. Comparing different sources may facilitate the learning process for normal users by creating the patterns of the security knowledge gained from different sources. Prior studies neither systematically analysed the wide range of digital sources nor provided any standardisation in analysing the trending topics from recent security texts. Moreover, existing topic modelling methods are not capable of identifying the cybersecurity concepts completely and the generated topics considerably overlap. To address this issue, we propose a semi-automated classification method to generate comprehensive security categories to analyse trending topics. We further compare the identified 16 security categories across different sources based on their popularity and impact. We have revealed several surprising findings as follows: (1) The impact reflected from cybersecurity texts strongly correlates with the monetary loss caused by cybercrimes, (2) security blogs have produced the context of cybersecurity most intensively, and (3) websites deliver security information without caring about timeliness much.<\/jats:p>","DOI":"10.1145\/3483332","type":"journal-article","created":{"date-parts":[[2021,10,29]],"date-time":"2021-10-29T15:04:03Z","timestamp":1635519843000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Analysis of Trending Topics and Text-based Channels of Information Delivery in Cybersecurity"],"prefix":"10.1145","volume":"22","author":[{"given":"Tingmin","family":"Wu","sequence":"first","affiliation":[{"name":"Swinburne University of Technology, Australia and CSIRO\u2019s Data61, Australia"}]},{"given":"Wanlun","family":"Ma","sequence":"additional","affiliation":[{"name":"Swinburne University of Technology, Australia"}]},{"given":"Sheng","family":"Wen","sequence":"additional","affiliation":[{"name":"Swinburne University of Technology, Australia"}]},{"given":"Xin","family":"Xia","sequence":"additional","affiliation":[{"name":"Monash University, Australia"}]},{"given":"Cecile","family":"Paris","sequence":"additional","affiliation":[{"name":"CSIRO\u2019s Data61, Australia"}]},{"given":"Surya","family":"Nepal","sequence":"additional","affiliation":[{"name":"CSIRO\u2019s Data61, Australia"}]},{"given":"Yang","family":"Xiang","sequence":"additional","affiliation":[{"name":"Swinburne University of Technology, Australia"}]}],"member":"320","published-online":{"date-parts":[[2021,10,29]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.11591\/ijeecs.v10.i1.pp371-379"},{"key":"e_1_3_3_3_2","volume-title":"Proceedings of the 8th USENIX Workshop on Free and Open Communications on the Internet (FOCI\u201918)","author":"Abu-Salma Ruba","year":"2018","unstructured":"Ruba Abu-Salma, Elissa M. Redmiles, Blase Ur, and Miranda Wei. 2018. Exploring user mental models of end-to-end encrypted communication tools. In Proceedings of the 8th USENIX Workshop on Free and Open Communications on the Internet (FOCI\u201918)."},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2005.22"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.5555\/2534766.2534789"},{"key":"e_1_3_3_6_2","article-title":"Ransomware Attack Still Looms in Australia as Government Warns WannaCry Threat Not Over","author":"Belot Henry","year":"2020","unstructured":"Henry Belot and ABC News Stephanie Borys. cited May 2020. Ransomware Attack Still Looms in Australia as Government Warns WannaCry Threat Not Over. Retrieved from https:\/\/www.abc.net.au\/news\/2017-05-15\/ransomware-attack-to-hit-victims-in-australia-government-says\/8526346.","journal-title":"https:\/\/www.abc.net.au\/news\/2017-05-15\/ransomware-attack-to-hit-victims-in-australia-government-says\/8526346"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2015.01.039"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0895-4356(00)00314-0"},{"key":"e_1_3_3_9_2","doi-asserted-by":"crossref","unstructured":"David M. Blei and John D. Lafferty. 2009. Topic models. In Text Mining . Chapman and Hall\/CRC 101\u2013124.","DOI":"10.1201\/9781420059458-12"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.5555\/944919.944937"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1093\/cybsec\/tyz005"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1037\/1082-989X.7.3.338"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/2663876.2663883"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1147"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290607.3312935"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2017.03.007"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-4003"},{"key":"e_1_3_3_19_2","doi-asserted-by":"crossref","unstructured":"Robert E. Crossler and France B\u00e9langer. 2017. The mobile privacy-security knowledge gap model: Understanding behaviors. In Proceedings of the Hawaii International Conference on System Sciences .","DOI":"10.24251\/HICSS.2017.491"},{"journal-title":"https:\/\/www.cybsafe.com\/press-releases\/human-error-to-blame-for-9-in-10-uk-cyber-data-breaches-in-2019\/","article-title":"Human error to blame for 9 in 10 UK cyber data breaches in 2019","key":"e_1_3_3_20_2","unstructured":"CybSafe. Human error to blame for 9 in 10 UK cyber data breaches in 2019. Retrieved February 2020 from https:\/\/www.cybsafe.com\/press-releases\/human-error-to-blame-for-9-in-10-uk-cyber-data-breaches-in-2019\/."},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.5555\/3235838.3235851"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/2660267.2660271"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173575"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1002\/sec.1657"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.5555\/3235895.3235902"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2017.3301619"},{"issue":"2","key":"e_1_3_3_27_2","first-page":"3","article-title":"Do stories help people adopt two-factor authentication?","volume":"1","author":"Fennell Chris","year":"2019","unstructured":"Chris Fennell and Rick Wash. 2019. Do stories help people adopt two-factor authentication? Studies 1, 2 (2019), 3.","journal-title":"Studies"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.5555\/3235895.3235904"},{"key":"e_1_3_3_29_2","article-title":"The Impact of cybersecurity over the Last 5 Years","author":"CCSIR Cyber Security Research Center from Romania","year":"2020","unstructured":"Cyber Security Research Center from Romania CCSIR. cited Feb 2020. The Impact of cybersecurity over the Last 5 Years. Retrieved from https:\/\/def.camp\/impact-cybersecurity-five-years\/.","journal-title":"https:\/\/def.camp\/impact-cybersecurity-five-years\/"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/NBiS.2015.41"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3243767"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-019-09753-2"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.heliyon.2017.e00346"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00092"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.5555\/3277203.3277285"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2015.7081810"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.21105\/joss.01556"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.5555\/3235866.3235893"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcss.2014.02.005"},{"key":"e_1_3_3_40_2","article-title":"IC3: Total Damage Caused by Reported Cyber Crime 2001\u20132020","author":"Johnson Joseph","unstructured":"Joseph Johnson. IC3: Total Damage Caused by Reported Cyber Crime 2001\u20132020. Retrieved April 2020 from https:\/\/www.statista.com\/statistics\/267132\/total-damage-caused-by-by-cyber-crime-in-the-us\/.","journal-title":"https:\/\/www.statista.com\/statistics\/267132\/total-damage-caused-by-by-cyber-crime-in-the-us\/"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.55"},{"key":"e_1_3_3_42_2","article-title":"Cyber-attacks a Bigger Threat Than Al Qaeda, Officials Say","author":"Dilanian Ken","unstructured":"Ken Dilanian. Los Angeles Times. Cyber-attacks a Bigger Threat Than Al Qaeda, Officials Say. Retrieved April 2020 from https:\/\/www.latimes.com\/world\/la-xpm-2013-mar-12-la-fg-worldwide-threats-20130313-story.html.","journal-title":"https:\/\/www.latimes.com\/world\/la-xpm-2013-mar-12-la-fg-worldwide-threats-20130313-story.html"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.5555\/3241189.3241293"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.3390\/info11030152"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/2976749.2978315"},{"key":"e_1_3_3_46_2","first-page":"681","volume-title":"Artificial Intelligence and Statistics","author":"Mei Shike","year":"2015","unstructured":"Shike Mei and Xiaojin Zhu. 2015. The security of latent dirichlet allocation. In Artificial Intelligence and Statistics. 681\u2013689."},{"key":"e_1_3_3_47_2","article-title":"Obama Administration Announces Measures to Punish Russia for 2016 Election Interference","author":"Ryan Ellen Nakashima, Missy","unstructured":"Ellen Nakashima, Missy Ryan, and Karen DeYoung. The Washington Post. Obama Administration Announces Measures to Punish Russia for 2016 Election Interference. Retrieved May 2020 from https:\/\/www.washingtonpost.com\/world\/national-security\/obama-administration-announces-measures-to-punish-russia-for-2016-election-interference\/2016\/12\/29\/311db9d6-cdde-11e6-a87f-b917067331bb_story.html.","journal-title":"https:\/\/www.washingtonpost.com\/world\/national-security\/obama-administration-announces-measures-to-punish-russia-for-2016-election-interference\/2016\/12\/29\/311db9d6-cdde-11e6-a87f-b917067331bb_story.html"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3055305.3055309"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2005.249"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300579"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.5555\/3291228.3291235"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2015.01.004"},{"key":"e_1_3_3_53_2","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1007\/3-540-32394-5_20","volume-title":"Knowledge Mining","author":"Pazienza Maria Teresa","year":"2005","unstructured":"Maria Teresa Pazienza, Marco Pennacchiotti, and Fabio Massimo Zanzotto. 2005. Terminology extraction: An analysis of linguistic and statistical approaches. In Knowledge Mining. Springer, 255\u2013279."},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300748"},{"issue":"1","key":"e_1_3_3_55_2","first-page":"121","article-title":"Identifying patterns in informal sources of security information","volume":"1","author":"Rader Emilee","year":"2015","unstructured":"Emilee Rader and Rick Wash. 2015. Identifying patterns in informal sources of security information. J. Cybersecur. 1, 1 (2015), 121\u2013144.","journal-title":"J. Cybersecur."},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/2335356.2335364"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/2976749.2978307"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025673"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2016.24"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219166.3219185"},{"key":"e_1_3_3_61_2","volume-title":"Proceedings of the IEEE Security & Privacy Workshop on Technology and Consumer Protection (ConPro\u201918)","author":"Redmiles Elissa M.","year":"2018","unstructured":"Elissa M. Redmiles, Miraida Morales, Lisa Maszkiewicz, Rock Stevens, Everest Liu, Dhruv Kuchhal, and Michelle L. Mazurek. 2018. First steps toward measuring the readability of security advice. In Proceedings of the IEEE Security & Privacy Workshop on Technology and Consumer Protection (ConPro\u201918)."},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/msp.2017.3681050"},{"key":"e_1_3_3_63_2","first-page":"45","volume-title":"Proceedings of the LREC Workshop on New Challenges for NLP Frameworks","author":"\u0158eh\u016f\u0159ek Radim","year":"2010","unstructured":"Radim \u0158eh\u016f\u0159ek and Petr Sojka. 2010. Software framework for topic modelling with large corpora. In Proceedings of the LREC Workshop on New Challenges for NLP Frameworks. ELRA, 45\u201350. http:\/\/is.muni.cz\/publication\/884893\/en."},{"key":"e_1_3_3_64_2","unstructured":"Leonard Richardson. 2007. Beautiful soup documentation. Dosegljivo: https:\/\/www.crummy.com\/software\/BeautifulSoup\/bs4\/doc\/. [Dostopano: 7. 7. 2018] (2007)."},{"key":"e_1_3_3_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/2684822.2685324"},{"key":"e_1_3_3_66_2","first-page":"103","article-title":"Grey literature","volume":"2","author":"Rothstein Hannah R.","year":"2009","unstructured":"Hannah R. Rothstein and Sally Hopewell. 2009. Grey literature. Handb. Res. Synth. Meta-anal. 2 (2009), 103\u2013125.","journal-title":"Handb. Res. Synth. Meta-anal."},{"key":"e_1_3_3_67_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2015.12.037"},{"key":"e_1_3_3_68_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2018.12.011"},{"key":"e_1_3_3_69_2","doi-asserted-by":"publisher","DOI":"10.5555\/3235866.3235889"},{"key":"e_1_3_3_70_2","article-title":"If Your Employees Aren\u2019t Learning from Your Security Training, Are You Really Teaching?","author":"Schultz Tyler","unstructured":"Tyler Schultz. If Your Employees Aren\u2019t Learning from Your Security Training, Are You Really Teaching? Retrieved February 2020 from https:\/\/www.infosecinstitute.com\/blog\/if-your-employees-arent-learning-from-your-security-training-are-you-really-teaching\/.","journal-title":"https:\/\/www.infosecinstitute.com\/blog\/if-your-employees-arent-learning-from-your-security-training-are-you-really-teaching\/"},{"key":"e_1_3_3_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/PST.2017.00027"},{"key":"e_1_3_3_72_2","doi-asserted-by":"publisher","DOI":"10.24251\/HICSS.2017.604"},{"key":"e_1_3_3_73_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2016.04.003"},{"key":"e_1_3_3_74_2","volume-title":"Card Sorting: Designing Usable Categories","author":"Spencer Donna","year":"2009","unstructured":"Donna Spencer. 2009. Card Sorting: Designing Usable Categories. Rosenfeld Media."},{"key":"e_1_3_3_75_2","doi-asserted-by":"publisher","DOI":"10.5555\/3277203.3277250"},{"key":"e_1_3_3_76_2","doi-asserted-by":"publisher","DOI":"10.5555\/1855768.1855793"},{"key":"e_1_3_3_77_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2017.09.001"},{"key":"e_1_3_3_78_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2921343"},{"key":"e_1_3_3_79_2","doi-asserted-by":"publisher","DOI":"10.1145\/1837110.1837125"},{"key":"e_1_3_3_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300338"},{"key":"e_1_3_3_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/3320269.3384761"},{"key":"e_1_3_3_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/2961111.2962606"},{"key":"e_1_3_3_83_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2018.2834227"},{"key":"e_1_3_3_84_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11390-016-1672-0"},{"key":"e_1_3_3_85_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300424"},{"key":"e_1_3_3_86_2","doi-asserted-by":"publisher","DOI":"10.5555\/3291228.3291245"},{"key":"e_1_3_3_87_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSEC.2019.2897834"}],"container-title":["ACM Transactions on Internet Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3483332","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3483332","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:18:57Z","timestamp":1750191537000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3483332"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,29]]},"references-count":86,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,5,31]]}},"alternative-id":["10.1145\/3483332"],"URL":"https:\/\/doi.org\/10.1145\/3483332","relation":{},"ISSN":["1533-5399","1557-6051"],"issn-type":[{"type":"print","value":"1533-5399"},{"type":"electronic","value":"1557-6051"}],"subject":[],"published":{"date-parts":[[2021,10,29]]},"assertion":[{"value":"2021-02-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-10-29","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}