{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T03:58:10Z","timestamp":1777694290784,"version":"3.51.4"},"reference-count":114,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2023,2,1]],"date-time":"2023-02-01T00:00:00Z","timestamp":1675209600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Integrated Computer-Aided Engineering"],"published-print":{"date-parts":[[2024,2]]},"abstract":"<jats:p>Private and military troll factories (facilities used to spread rumours in online social media) are currently proliferating around the world. By their very nature, they are obscure companies whose internal workings are largely unknown, apart from leaks to the press. They are even more concealed when it comes to their underlying technology. At least in a broad sense, it is believed that there are two main tasks performed by a troll factory: sowing and spreading. The first is to create and, more importantly, maintain a social network that can be used for the spreading task. It is then a wicked long-term activity, subject to all sorts of problems. As an attempt to make this perspective a little clearer, this paper uses exploratory design science research to produce artefacts that could be applied to online rumour spreading in social media. Then, as a hypothesis: it is possible to design a fully automated social media agent capable of sowing a social network on microblogging platforms. The expectation is that it will be possible to identify common opportunities and difficulties in the development of such tools, which in turn will allow an evaluation of the technology, but above all the level of automation of these facilities. The research is based on a general domain Twitter corpus with 4M[Formula: see text] tokens and on ChatGPT, and discusses both knowledge-based and deep learning approaches for smooth tweet generation. These explorations suggest that for the current, widespread and publicly available NLP technology, troll factories work like a call centre; i.e. humans assisted by more or less sophisticated computing tools (often called cyborgs).<\/jats:p>","DOI":"10.3233\/ica-230720","type":"journal-article","created":{"date-parts":[[2023,10,13]],"date-time":"2023-10-13T12:09:11Z","timestamp":1697198951000},"page":"95-115","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":8,"title":["An exploratory design science research on troll factories"],"prefix":"10.1177","volume":"31","author":[{"given":"Francisco S.","family":"Marcondes","sequence":"first","affiliation":[{"name":"ALGORITMI Research Centre\/LASI, University of Minho, Braga, Portugal"}]},{"given":"Jos\u00e9 Jo\u00e3o","family":"Almeida","sequence":"additional","affiliation":[{"name":"ALGORITMI Research Centre\/LASI, University of Minho, Braga, Portugal"}]},{"given":"Paulo","family":"Novais","sequence":"additional","affiliation":[{"name":"ALGORITMI Research Centre\/LASI, University of Minho, Braga, Portugal"}]}],"member":"179","published-online":{"date-parts":[[2023,2,1]]},"reference":[{"key":"bibr1-ICA-230720","first-page":"397","author":"Marcondes FS","year":"2021","journal-title":"International Conference on Intelligent Data Engineering and Automated Learning"},{"key":"bibr2-ICA-230720","unstructured":"PomerantsevP. This is not propaganda: Adventures in the war against reality. PublicAffairs; 2019."},{"key":"bibr3-ICA-230720","unstructured":"NasrineSA. Savvy Digital Citizenship: How to Master the Social Media World of Trolls, Bots, and Propaganda. eBook Kindle; 2021."},{"key":"bibr4-ICA-230720","unstructured":"RuggieroP. Re-Discovering the Cognitive Battlefield: What\u2019s Next? 2021. NATO StratCom COE; #RigaStratComDialogue. Available from: https:\/\/www.youtube.com\/watch?v=AQoASu2ihEE."},{"key":"bibr5-ICA-230720","first-page":"83","author":"Peffers K","year":"2006","journal-title":"Proceedings of the First International Conference on Design Science Research in Information Systems and Technology (DESRIST 2006)"},{"key":"bibr6-ICA-230720","doi-asserted-by":"publisher","DOI":"10.2307\/25148625"},{"key":"bibr7-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1109\/52.43044"},{"key":"bibr8-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1016\/j.paid.2014.01.016"},{"key":"bibr9-ICA-230720","doi-asserted-by":"crossref","unstructured":"StanleyJ. How propaganda works. Princeton University Press; 2015.","DOI":"10.1515\/9781400865802"},{"key":"bibr10-ICA-230720","doi-asserted-by":"crossref","unstructured":"JungherrA RiveroG Gayo-AvelloD. Retooling politics: How digital media are shaping democracy. Cambridge University Press; 2020.","DOI":"10.1017\/9781108297820"},{"key":"bibr11-ICA-230720","unstructured":"du CluzelF. Cognitive Warfare. NATO; 2021."},{"key":"bibr12-ICA-230720","unstructured":"IretonC PosettiJ. Journalism, fake news & disinformation: handbook for journalism education and training. Unesco Publishing; 2018."},{"key":"bibr13-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijdrr.2022.103144"},{"key":"bibr14-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2021.104573"},{"issue":"7","key":"bibr15-ICA-230720","first-page":"1","volume":"11","author":"Burhan R","year":"2020","journal-title":"Journal of Neurology & Neurophysiology"},{"issue":"1","key":"bibr16-ICA-230720","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-020-20314-w","volume":"12","author":"Lindstr\u00f6m B","year":"2021","journal-title":"Nature communications"},{"key":"bibr17-ICA-230720","doi-asserted-by":"crossref","unstructured":"HadnagyC FincherM. Phishing dark waters: The offensive and defensive sides of malicious Emails. John Wiley & Sons; 2015.","DOI":"10.1002\/9781119183624"},{"key":"bibr18-ICA-230720","unstructured":"LanierJ. Ten arguments for deleting your social media accounts right now. Random House; 2018."},{"key":"bibr19-ICA-230720","unstructured":"JacobJ. Modi Vs Gandhi: Twitter Analytics Reveal Fake Followers Plague Both Handles. 2017. Available from: https:\/\/www.boomlive.in\/modi-vs-gandhi-twitter-analytics-reveal-fake-followers-plague-both-handles\/."},{"key":"bibr20-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1145\/2818717"},{"key":"bibr21-ICA-230720","unstructured":"FishkinR. SparkToro & Followerwonk Joint Twitter Analysis: 19.42% of Active Accounts Are Fake or Spam. 2022. Available from: https:\/\/sparktoro.com\/blog\/sparktoro-followerwonk-joint-twitter-analysis-19-42-of-active-accounts-are-fake-or-spam\/."},{"key":"bibr22-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1057\/s41599-020-00550-7"},{"key":"bibr23-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.11164"},{"key":"bibr24-ICA-230720","unstructured":"SmithA. Does YouTube\u2019s algorithm lead to radicalization? PCMag; 2019. Available from: https:\/\/www.pcmag.com\/news\/does-youtubes-algorithm-lead-to-radicalization."},{"key":"bibr25-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1080\/15205436.2014.964871"},{"key":"bibr26-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2021.02.007"},{"key":"bibr27-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-020-01452-z"},{"key":"bibr28-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-020-0833-x"},{"key":"bibr29-ICA-230720","unstructured":"TreygerE CheravitchJ CohenR. Russian Disinformation Efforts on Social Media. Rand Corp Santa Monica CA; 2022."},{"key":"bibr30-ICA-230720","doi-asserted-by":"publisher","DOI":"10.52306\/01010418ZDCD5438"},{"key":"bibr31-ICA-230720","unstructured":"SydelL. We Tracked Down A Fake-News Creator In The Suburbs. Here\u2019s What We Learned. 2016. Available from: https:\/\/www.npr.org\/sections\/alltechconsidered\/2016\/11\/23\/503146770\/npr-finds-the-head-of-a-covert-fake-news-operation-in-the-suburbs?t=1578069801587&t=1599905256928."},{"key":"bibr32-ICA-230720","unstructured":"MusilS. Cyberattack on Twitter targeted Epilepsy Foundation with strobing images. 2019. Available from: https:\/\/www.cnet.com\/tech\/services-and-software\/cyberattack-on-twitter-targeted-epilepsy-foundation-with-strobing-images\/."},{"key":"bibr33-ICA-230720","author":"Woolley SC","year":"2018","journal-title":"Oxford Studies in Digital Politics"},{"key":"bibr34-ICA-230720","unstructured":"ButowE HermanJ LiuS RobinsonA AlltonM. Ultimate Guide to Social Media Marketing. Entrepreneur Press; 2020."},{"key":"bibr35-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12702"},{"key":"bibr36-ICA-230720","unstructured":"SantiagoD. A Tempestade Perfeita: como a extrema direita regressou \u00c3  pen\u00ednsula ib\u00e9rica. Oficina do Livro; 2021."},{"key":"bibr37-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1007\/s11023-019-09498-3"},{"key":"bibr38-ICA-230720","first-page":"1","author":"Echeverria J","year":"2017","journal-title":"Proceedings of the 2017 IEEE\/ACM international conference on advances in social networks analysis and mining 2017"},{"key":"bibr39-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1590\/S0104-530X2013005000014"},{"key":"bibr40-ICA-230720","unstructured":"SimonHA. The sciences of the artificial. MIT press; 2019."},{"key":"bibr41-ICA-230720","doi-asserted-by":"publisher","DOI":"10.2307\/3100886"},{"key":"bibr42-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1016\/j.techsoc.2019.101193"},{"key":"bibr43-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2005.08.004"},{"key":"bibr44-ICA-230720","unstructured":"TEC-SHS. Technology Readiness Levels Handbook for Space Applications. Technical report. ESA; 2008."},{"key":"bibr45-ICA-230720","unstructured":"EilamE. Reversing: secrets of reverse engineering. John Wiley & Sons; 2011."},{"key":"bibr46-ICA-230720","unstructured":"LemesDdO. F\u00e1bula PXP \u2013 a t\u00e9cnica de Programa\u00e7\u00e3o Explorat\u00f3ria (PXP): projetos de cria\u00e7\u00e3o e desenvolvimento de jogos digitais. PhD thesis. M\u00eddias Digitais. 2015."},{"key":"bibr47-ICA-230720","doi-asserted-by":"crossref","unstructured":"PopperK. The logic of scientific discovery. Routledge; 2005.","DOI":"10.4324\/9780203994627"},{"key":"bibr48-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1007\/BF01405730"},{"key":"bibr49-ICA-230720","unstructured":"ConklinJ. Dialogue mapping: Building shared understanding of wicked problems. John Wiley & Sons, Inc. 2005."},{"key":"bibr50-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12818"},{"key":"bibr51-ICA-230720","doi-asserted-by":"publisher","DOI":"10.3233\/ICA-200627"},{"key":"bibr52-ICA-230720","unstructured":"CombsTD BrownRA. Digital Behaviorism Design. Boundless Mind; 2018."},{"key":"bibr53-ICA-230720","first-page":"24","author":"Marcondes FS","year":"2022","journal-title":"International Work-Conference on the Interplay Between Natural and Artificial Computation"},{"key":"bibr54-ICA-230720","first-page":"203","author":"Barbosa MA","year":"2023","journal-title":"International Symposium on Distributed Computing and Artificial Intelligence"},{"key":"bibr55-ICA-230720","first-page":"170","author":"Marcondes FS","journal-title":"World Conference on Information Systems and Technologies"},{"key":"bibr56-ICA-230720","doi-asserted-by":"publisher","DOI":"10.3233\/ICA-210669"},{"key":"bibr57-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2012.2192811"},{"key":"bibr58-ICA-230720","author":"Marcondes FS","year":"2023","journal-title":"the 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023)"},{"key":"bibr59-ICA-230720","first-page":"374","author":"Marcondes FS","year":"2018","journal-title":"International Conference on Intelligent Data Engineering and Automated Learning"},{"key":"bibr60-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1007\/BF01692511"},{"key":"bibr61-ICA-230720","doi-asserted-by":"publisher","DOI":"10.3233\/ICA-200626"},{"key":"bibr62-ICA-230720","first-page":"1","author":"Li H","year":"2022","journal-title":"Integrated Computer-Aided Engineering"},{"key":"bibr63-ICA-230720","doi-asserted-by":"publisher","DOI":"10.3233\/ICA-210667"},{"key":"bibr64-ICA-230720","doi-asserted-by":"publisher","DOI":"10.3233\/ICA-210664"},{"key":"bibr65-ICA-230720","doi-asserted-by":"publisher","DOI":"10.21595\/jve.2017.18924"},{"key":"bibr66-ICA-230720","author":"Garfinkel SL","year":"2015","journal-title":"National Institute of Standards and Technology (NIST)"},{"key":"bibr67-ICA-230720","first-page":"1","author":"Demertzis K","year":"2022","journal-title":"Integrated Computer-Aided Engineering"},{"key":"bibr68-ICA-230720","unstructured":"JurafskyD MartinJH. Speech and language processing: An introduction to natural language processing. Wiley; 2006."},{"key":"bibr69-ICA-230720","unstructured":"LoperE BirdS. Nltk: The natural language toolkit. arXiv preprint cs\/0205028. 2002."},{"key":"bibr70-ICA-230720","unstructured":"GhoshS GunningD. Natural Language Processing Fundamentals: Build intelligent applications that can interpret the human language to deliver impactful results. Packt Publishing Ltd; 2019."},{"key":"bibr71-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0144296"},{"key":"bibr72-ICA-230720","unstructured":"GimpelK SchneiderN O\u2019ConnorB DasD MillsD EisensteinJ HeilmanM YogatamaD FlaniganJ SmithNA. Part-of-speech tagging for twitter: Annotation, features, and experiments. Carnegie-Mellon Univ Pittsburgh Pa School of Computer Science; 2010."},{"key":"bibr73-ICA-230720","author":"Wallace R","year":"2003","journal-title":"Alice AI Foundation"},{"key":"bibr74-ICA-230720","unstructured":"BocklischT FaulknerJ PawlowskiN NicholA. Rasa: Open source language understanding and dialogue management. arXiv preprint arXiv:171205181. 2017."},{"key":"bibr75-ICA-230720","first-page":"223","author":"Faria H","year":"2022","journal-title":"International Conference on Intelligent Data Engineering and Automated Learning"},{"key":"bibr76-ICA-230720","doi-asserted-by":"crossref","unstructured":"R\u00fchlemannC. Corpus linguistics for pragmatics: A guide for research. Routledge; 2018.","DOI":"10.4324\/9780429451072"},{"key":"bibr77-ICA-230720","first-page":"1169","author":"Derczynski L","year":"2016","journal-title":"Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers"},{"key":"bibr78-ICA-230720","unstructured":"LiuY ZhuY CheW QinB SchneiderN SmithNA. Parsing tweets into universal dependencies. arXiv preprint arXiv:180408228. 2018."},{"key":"bibr79-ICA-230720","unstructured":"Esp\u00f3sitoF. Twitter restricts its search to registered users while Musk suggests Fleets are coming back. 2023. Available from: https:\/\/9to5mac.com\/2023\/04\/26\/twitter-restricts-search-fleets-coming-back\/."},{"key":"bibr80-ICA-230720","author":"Marcondes FS","year":"2020","journal-title":"STAIRS@ ECAI"},{"key":"bibr81-ICA-230720","volume":"30","author":"Vaswani A","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"bibr82-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i8.20801"},{"key":"bibr83-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1109\/APSIPAASC47483.2019.9023353"},{"key":"bibr84-ICA-230720","first-page":"1","author":"Rohman YA","year":"2019","journal-title":"2019 3rd International Conference on Informatics and Computational Sciences (ICICoS)"},{"key":"bibr85-ICA-230720","doi-asserted-by":"publisher","DOI":"10.58729\/1941-6679.1460"},{"key":"bibr86-ICA-230720","unstructured":"WiggersK. There\u2019s no reason to panic over WormGPT. 2023. Available from: https:\/\/techcrunch.com\/2023\/08\/01\/theres-no-reason-to-panic-over-wormgpt\/."},{"key":"bibr87-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1145\/3144592.3144598"},{"key":"bibr88-ICA-230720","unstructured":"WhiteJ FuQ HaysS SandbornM OleaC GilbertH ElnasharA Spencer-SmithJ SchmidtDC. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:230211382. 2023."},{"key":"bibr89-ICA-230720","first-page":"1877","volume":"33","author":"Brown T","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"bibr90-ICA-230720","first-page":"1","author":"D\u00edaz H","year":"2023","journal-title":"Integrated Computer-Aided Engineering"},{"key":"bibr91-ICA-230720","unstructured":"RogersK. Bots tweeting about Toronto\u2019s Sidewalk Labs project raise disinformation red flags. 2019. Available from: https:\/\/www.cbc.ca\/news\/science\/twitter-botnet-sidewalklabs-canada-election-1.5206900."},{"key":"bibr92-ICA-230720","unstructured":"BarojanD. Influence For Sale: Bot Shopping on the Darknet. 2017. Available from: https:\/\/medium.com\/dfrlab\/influence-for-sale-bot-shopping-on-the-darknet-1c9ddfb3d8e6."},{"key":"bibr93-ICA-230720","unstructured":"HuE. What is Information Warfare? NATO StratCom COE; Expert seminar \u2018Social Media, Communication and Security\u2019. 2018. Available from: https:\/\/www.youtube.com\/watch?v=H0ThZNFmz8w."},{"key":"bibr94-ICA-230720","doi-asserted-by":"publisher","DOI":"10.21659\/rupkatha.v12n5.rioc1s21n2"},{"key":"bibr95-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1007\/s10732-009-9107-5"},{"key":"bibr96-ICA-230720","doi-asserted-by":"publisher","DOI":"10.2307\/2102968"},{"key":"bibr97-ICA-230720","unstructured":"GabbayDM GuenthnerF, et al. Handbook of philosophical logic. vol. 3. Springer; 1986."},{"key":"bibr98-ICA-230720","unstructured":"CMMI Product Team. CMMI\u00ae for Development, Version 1.3. SEI, CMU; 2010."},{"key":"bibr99-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0241045"},{"key":"bibr100-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1109\/PDP2018.2018.00077"},{"key":"bibr101-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1145\/3314545.3314562"},{"key":"bibr102-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1037\/h0043158"},{"key":"bibr103-ICA-230720","volume":"16","author":"Degenne A","year":"2009","journal-title":"Redes"},{"key":"bibr104-ICA-230720","doi-asserted-by":"publisher","DOI":"10.31887\/DCNS.2016.18.1\/wschultz"},{"key":"bibr105-ICA-230720","author":"Marcondes FS","year":"2022","journal-title":"Braga Hate Speech Conference: Book of Abstracts"},{"key":"bibr106-ICA-230720","first-page":"1638","author":"Akbik A","year":"2018","journal-title":"Proceedings of the 27th international conference on computational linguistics"},{"key":"bibr107-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2015.2489653"},{"key":"bibr108-ICA-230720","doi-asserted-by":"crossref","unstructured":"HollensteinN de la TorreA LangerN ZhangC. CogniVal: A framework for cognitive word embedding evaluation. arXiv preprint arXiv:190909001. 2019.","DOI":"10.18653\/v1\/K19-1050"},{"key":"bibr109-ICA-230720","first-page":"607","author":"Carvalho V","year":"2012","journal-title":"Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies"},{"key":"bibr110-ICA-230720","doi-asserted-by":"publisher","DOI":"10.7312\/harg18876-002"},{"key":"bibr111-ICA-230720","doi-asserted-by":"publisher","DOI":"10.1140\/epjds\/s13688-018-0178-0"},{"key":"bibr112-ICA-230720","unstructured":"WelchC. Twitter halts plan to remove inactive accounts until it can memorialize dead users. The Verge. 2019. Available from: https:\/\/www.theverge.com\/2019\/11\/27\/20986084\/twitter-inactive-accounts-usernames-memorialize-deceased-users-not-removing."},{"key":"bibr113-ICA-230720","unstructured":"JiangH HuaY BeefermanD RoyD. Annotating the Tweebank Corpus on Named Entity Recognition and Building NLP Models for Social Media Analysis. arXiv preprint arXiv:220107281. 2022."},{"key":"bibr114-ICA-230720","doi-asserted-by":"crossref","unstructured":"NguyenDQ VuT NguyenAT. BERTweet: A pre-trained language model for English Tweets. arXiv preprint arXiv: 200510200. 2020.","DOI":"10.18653\/v1\/2020.emnlp-demos.2"}],"container-title":["Integrated Computer-Aided Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/ICA-230720","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.3233\/ICA-230720","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/ICA-230720","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:14:50Z","timestamp":1777454090000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.3233\/ICA-230720"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,1]]},"references-count":114,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,2]]}},"alternative-id":["10.3233\/ICA-230720"],"URL":"https:\/\/doi.org\/10.3233\/ica-230720","relation":{},"ISSN":["1069-2509","1875-8835"],"issn-type":[{"value":"1069-2509","type":"print"},{"value":"1875-8835","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,1]]}}}