{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,6]],"date-time":"2026-02-06T02:43:03Z","timestamp":1770345783189,"version":"3.49.0"},"reference-count":61,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2021,7,1]],"date-time":"2021-07-01T00:00:00Z","timestamp":1625097600000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Big Data &amp; Society"],"published-print":{"date-parts":[[2021,7]]},"abstract":"<jats:p> Social bots \u2013 partially or fully automated accounts on social media platforms \u2013 have not only been widely discussed, but have also entered political, media and research agendas. However, bot detection is not an exact science. Quantitative estimates of bot prevalence vary considerably and comparative research is rare. We show that findings on the prevalence and activity of bots on Twitter depend strongly on the methods used to identify automated accounts. We search for bots in political discourses on Twitter, using three different bot detection methods: Botometer, Tweetbotornot and \u201cheavy automation\u201d. We drew a sample of 122,884 unique user Twitter accounts that had produced 263,821 tweets contributing to five political discourses in five Western democracies. While all three bot detection methods classified accounts as bots in all our cases, the comparison shows that the three approaches produce very different results. We discuss why neither manual validation nor triangulation resolves the basic problems, and conclude that social scientists studying the influence of social bots on (political) communication and discourse dynamics should be careful with easy-to-use methods, and consider interdisciplinary research. <\/jats:p>","DOI":"10.1177\/20539517211033566","type":"journal-article","created":{"date-parts":[[2021,8,24]],"date-time":"2021-08-24T06:09:59Z","timestamp":1629785399000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":69,"title":["Bot, or not? Comparing three methods for detecting social bots  in five political discourses"],"prefix":"10.1177","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1283-2659","authenticated-orcid":false,"given":"Franziska","family":"Martini","sequence":"first","affiliation":[{"name":"Weizenbaum Institute for the Networked Society, Freie Universit\u00e4t Berlin, Germany"}]},{"given":"Paul","family":"Samula","sequence":"additional","affiliation":[{"name":"Weizenbaum Institute for the Networked Society, Freie Universit\u00e4t Berlin, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5263-4812","authenticated-orcid":false,"given":"Tobias R","family":"Keller","sequence":"additional","affiliation":[{"name":"Digital Media Research Centre, Queensland University of Technology, Australia; gfs.bern, Bern, Switzerland (present)"}]},{"given":"Ulrike","family":"Klinger","sequence":"additional","affiliation":[{"name":"Weizenbaum Institute for the Networked Society, Freie Universit\u00e4t Berlin, Germany"}]}],"member":"179","published-online":{"date-parts":[[2021,8,23]]},"reference":[{"key":"bibr1-20539517211033566","first-page":"27","volume":"15","author":"Abrahams A","year":"2021","journal-title":"International Journal of Communication"},{"key":"bibr2-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1177\/2056305120939264"},{"key":"bibr3-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1177\/0894439317734157"},{"key":"bibr4-20539517211033566","doi-asserted-by":"crossref","unstructured":"Bessi A, Ferrara E (2016) Social bots distort the 2016 US presidential election online discussion. First Monday 21(11). Available at: https:\/\/ssrn.com\/abstract=2982233.","DOI":"10.5210\/fm.v21i11.7090"},{"key":"bibr5-20539517211033566","doi-asserted-by":"crossref","first-page":"26","DOI":"10.46300\/9107.2021.15.6","volume":"15","author":"Boichak O","year":"2021","journal-title":"International Journal of Communication"},{"key":"bibr6-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1145\/2076732.2076746"},{"key":"bibr7-20539517211033566","unstructured":"Botometer (2020) FAQ. Available at: https:\/\/botometer.osome.iu.edu\/faq (accessed 06 April 2021)."},{"key":"bibr8-20539517211033566","doi-asserted-by":"publisher","DOI":"10.2105\/AJPH.2018.304567"},{"key":"bibr9-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1080\/1369118X.2019.1637447"},{"key":"bibr10-20539517211033566","doi-asserted-by":"crossref","unstructured":"Bucher T (2014) About a Bot: Hoax, fake, performance Art. M\/C Journal, 17(3). Available at: http:\/\/journal.media-culture.org.au\/index.php\/mcjournal\/article\/view\/814.","DOI":"10.5204\/mcj.814"},{"key":"bibr11-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2016.0096"},{"key":"bibr12-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2012.75"},{"key":"bibr13-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2017.2681672"},{"key":"bibr14-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1145\/3041021.3055135"},{"key":"bibr15-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1145\/2872518.2889302"},{"key":"bibr16-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1145\/3274694.3274738"},{"key":"bibr17-20539517211033566","unstructured":"European Commission (2018) Action plan against disinformation. 05 December 2018. Available at: https:\/\/eeas.europa.eu\/sites\/default\/files\/action_plan_against_disinformation.pdf (accessed 31 May 2021)."},{"key":"bibr18-20539517211033566","doi-asserted-by":"publisher","DOI":"10.5210\/fm.v22i8.8005"},{"key":"bibr19-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-77332-2"},{"key":"bibr20-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1145\/2818717"},{"key":"bibr21-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1080\/10584609.2018.1477506"},{"key":"bibr22-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1007\/s13278-016-0331-3"},{"key":"bibr23-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1002\/poi3.184"},{"key":"bibr24-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-91521-0_32"},{"key":"bibr25-20539517211033566","doi-asserted-by":"crossref","unstructured":"Haunschild R, Bornmann L, Potnis D, et al. (2021) Investigating diffusion of scientific knowledge on Twitter: A study of topic networks of opioid publications. arXiv preprint arXiv:2101.11483.","DOI":"10.1162\/qss_a_00168"},{"key":"bibr26-20539517211033566","unstructured":"Hindman M, Barash V (2018) Disinformation, \u2018fake News\u2019 and influence campaigns on Twitter. Available at: http:\/\/apo.org.au\/system\/files\/195631\/apo-nid195631-1019641.pdf (accessed 26 October 2018)."},{"key":"bibr27-20539517211033566","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.2798311"},{"key":"bibr28-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1080\/19331681.2018.1448735"},{"key":"bibr29-20539517211033566","unstructured":"Kearney M (2018) Tweetbotornot: R package for detecting Twitter bots via machine learning [software package]. https:\/\/github.com\/mkearney\/Tweetbotornot (accessed 31 May 2021)."},{"key":"bibr30-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1080\/10584609.2018.1526238"},{"key":"bibr31-20539517211033566","first-page":"4932","volume":"10","author":"Kollanyi B","year":"2016","journal-title":"International Journal of Communication"},{"key":"bibr32-20539517211033566","unstructured":"Kollanyi B, Howard PN, Woolley SC (2016) Bots and automation over Twitter during the U.S. Election. COMPROP Data Memo 2016.4, Oxford, UK. Available at: http:\/\/comprop.oii.ox.ac.uk\/wp-content\/uploads\/sites\/89\/2016\/11\/Data-Memo-US-Election.pdf"},{"key":"bibr33-20539517211033566","unstructured":"Krebs B (2011) Twitter bots drown out anti-kremlin tweets. Krebs on security, 08 December. Available at: http:\/\/krebsonsecurity.com\/2011\/12\/twitter-bots-drown-out-anti-kremlin-tweets\/"},{"key":"bibr34-20539517211033566","doi-asserted-by":"publisher","DOI":"10.5210\/fm.v24i9.10213"},{"key":"bibr35-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1145\/3292522.3326015"},{"key":"bibr36-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1177\/1940161219872942"},{"key":"bibr37-20539517211033566","unstructured":"M\u00fcnch FV, Puschmann C, Thies B, et al. (2019) What makes a bot a bot? Exploring benign automation on Twitter. Conference presentation, Automating Communication in the Networked Society: Contexts, Consequences, Critique. Annual Conference of the German Communication Association\u2019s Division \u201cDigital Communication\u201d, 6\u20138 November 2019, Berlin. http:\/\/bit.ly\/whatabot"},{"key":"bibr38-20539517211033566","unstructured":"Musgrave S (2017) \u2018I Get called a Russian Bot 50 times a Day\u2019. Politico, 09 August Available at: https:\/\/www.politico.com\/magazine\/story\/2017\/08\/09\/twitter-trump-train-maga-echo-chamber-215470"},{"key":"bibr39-20539517211033566","unstructured":"Nelson M, Jaursch J (2020) Germany\u2019s new media treaty demands that platforms explain algorithms and stop discriminating. Can it deliver? Available at: https:\/\/algorithmwatch.org\/en\/new-media-treaty-germany\/ (accessed 31 May 2021)."},{"key":"bibr40-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1177\/2056305119863147"},{"key":"bibr41-20539517211033566","unstructured":"Neudert LM, Kollanyi B, Howard PN (2017) Junk news and bots during the German parliamentary election: What are German voters sharing over Twitter? COMPROP Data Memo 2017.7, Oxford, UK. Available at: http:\/\/comprop.oii.ox.ac.uk\/wp-content\/uploads\/sites\/89\/2017\/09\/ComProp_GermanElections_Sep2017v5.pdf"},{"key":"bibr42-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1016\/j.osnem.2019.100058"},{"key":"bibr43-20539517211033566","unstructured":"Pozzana I, Ferrara E (2018) Measuring bot and human behavioral dynamics. arXiv preprint arXiv:1802.04286."},{"key":"bibr44-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0241045"},{"key":"bibr45-20539517211033566","unstructured":"R Core Team (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Available at: http:\/\/www.R-project.org\/ (accessed 31 May 2021)."},{"key":"bibr46-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1080\/0960085X.2018.1560920"},{"key":"bibr47-20539517211033566","unstructured":"Sacharoff L (2018) Do bots have first amendment rights? Politico, 27 November. Available at: https:\/\/www.politico.com\/magazine\/story\/2018\/11\/27\/bots-first-amendment-rights-222689"},{"key":"bibr48-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3412698"},{"key":"bibr49-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1093\/ijpor\/12.1.3"},{"key":"bibr50-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0244309"},{"key":"bibr51-20539517211033566","doi-asserted-by":"crossref","unstructured":"Shao C, Ciampaglia GL, Varol O, et al. (2018) The spread of low-credibility content by social bots. arXiv preprint arXiv:1707.07592v4.","DOI":"10.1038\/s41467-018-06930-7"},{"key":"bibr52-20539517211033566","volume-title":"In: Australasian Conference on Information Systems (ACIS) 2017 Proceedings","volume":"89","author":"Stieglitz S","year":"2017"},{"key":"bibr53-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2016.183"},{"key":"bibr54-20539517211033566","first-page":"253","volume-title":"In: AAAI Spring Symposium Series","author":"Thieltges A","year":"2016"},{"key":"bibr55-20539517211033566","doi-asserted-by":"crossref","unstructured":"Varol O, Ferrara E, Davis CA, et al. (2017) Online human-bot interactions: Detection, estimation, and characterization. arXiv preprint arXiv:1703.03107.","DOI":"10.1609\/icwsm.v11i1.14871"},{"key":"bibr56-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1126\/science.aap9559"},{"key":"bibr57-20539517211033566","unstructured":"Wojcik S, Messing S, Smith A, et al. (2018) Bots in the Twittersphere. Report, Pew Research Center, 09 April. Available at: http:\/\/www.pewinternet.org\/2018\/04\/09\/bots-in-the-twittersphere\/ (accessed 26 October 2018)."},{"key":"bibr58-20539517211033566","doi-asserted-by":"publisher","DOI":"10.5210\/fm.v21i4.6161"},{"key":"bibr59-20539517211033566","first-page":"185","volume-title":"Computational Propaganda. Political Parties, Politicians, and Political Manipulation on Social Media","author":"Woolley SC","year":"2018"},{"key":"bibr60-20539517211033566","volume-title":"Computational Propaganda. Political Parties, Politicians, and Political Manipulation on Social Media","author":"Woolley SC","year":"2019"},{"key":"bibr61-20539517211033566","doi-asserted-by":"publisher","DOI":"10.1002\/hbe2.115"}],"container-title":["Big Data &amp; Society"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/20539517211033566","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/20539517211033566","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/20539517211033566","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,28]],"date-time":"2025-02-28T13:27:34Z","timestamp":1740749254000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/20539517211033566"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7]]},"references-count":61,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,7]]}},"alternative-id":["10.1177\/20539517211033566"],"URL":"https:\/\/doi.org\/10.1177\/20539517211033566","relation":{},"ISSN":["2053-9517","2053-9517"],"issn-type":[{"value":"2053-9517","type":"print"},{"value":"2053-9517","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7]]},"article-number":"20539517211033566"}}