{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,26]],"date-time":"2026-01-26T00:21:36Z","timestamp":1769386896208,"version":"3.49.0"},"reference-count":90,"publisher":"Emerald","issue":"5","license":[{"start":{"date-parts":[[2018,10,2]],"date-time":"2018-10-02T00:00:00Z","timestamp":1538438400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["INTR"],"published-print":{"date-parts":[[2018,10,2]]},"abstract":"<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title>\n<jats:p>The purpose of this paper is to explore the dark side of news community forums: the proliferation of opinion manipulation trolls. In particular, it explores the idea that a user who is called a troll by several people is likely to be one. It further demonstrates the utility of this idea for detecting accused and paid opinion manipulation trolls and their comments as well as for predicting the credibility of comments in news community forums.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title>\n<jats:p>The authors are aiming to build a classifier to distinguish trolls vs regular users. Unfortunately, it is not easy to get reliable training data. The authors solve this issue pragmatically: the authors assume that a user who is called a troll by several people is likely to be such, which are called accused trolls. Based on this assumption and on leaked reports about actual paid opinion manipulation trolls, the authors build a classifier to distinguish trolls vs regular users.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Findings<\/jats:title>\n<jats:p>The authors compare the profiles of paid trolls vs accused trolls vs non-trolls, and show that a classifier trained to distinguish accused trolls from non-trolls does quite well also at telling apart paid trolls from non-trolls.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Research limitations\/implications<\/jats:title>\n<jats:p>The troll detection works even for users with about 10 comments, but it achieves the best performance for users with a sizable number of comments in the forum, e.g. 100 or more. Yet, there is not such a limitation for troll comment detection.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Practical implications<\/jats:title>\n<jats:p>The approach would help forum moderators in their work, by pointing them to the most suspicious users and comments. It would be also useful to investigative journalists who want to find paid opinion manipulation trolls.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Social implications<\/jats:title>\n<jats:p>The authors can offer a better experience to online users by filtering out opinion manipulation trolls and their comments.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title>\n<jats:p>The authors propose a novel approach for finding paid opinion manipulation trolls and their posts.<\/jats:p>\n<\/jats:sec>","DOI":"10.1108\/intr-03-2017-0118","type":"journal-article","created":{"date-parts":[[2018,8,24]],"date-time":"2018-08-24T19:06:26Z","timestamp":1535137586000},"page":"1292-1312","source":"Crossref","is-referenced-by-count":34,"title":["The dark side of news community forums: opinion manipulation trolls"],"prefix":"10.1108","volume":"28","author":[{"given":"Todor","family":"Mihaylov","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tsvetomila","family":"Mihaylova","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Preslav","family":"Nakov","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Llu\u00eds","family":"M\u00e0rquez","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Georgi D.","family":"Georgiev","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ivan Kolev","family":"Koychev","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","reference":[{"key":"key2020100208591575700_ref001","first-page":"844","article-title":"PMI-cool at SemEval-2016 task 3: experiments with PMI and goodness polarity lexicons for community question answering","year":"2016"},{"key":"key2020100208591575700_ref002","first-page":"687","article-title":"Thread-level information for comment classification in community question answering","year":"2015"},{"issue":"5","key":"key2020100208591575700_ref003","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1108\/IntR-03-2012-0056","article-title":"Effects of complaint behaviour and service recovery satisfaction on consumer intentions to repurchase on the internet","volume":"24","year":"2014","journal-title":"Internet Research"},{"key":"key2020100208591575700_ref004","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/j.paid.2014.01.016","article-title":"Trolls just want to have fun","volume":"67","year":"2014","journal-title":"Personality and Individual Differences"},{"key":"key2020100208591575700_ref005","article-title":"Do not feel the trolls","year":"2010"},{"key":"key2020100208591575700_ref006","first-page":"675","article-title":"Information credibility on Twitter","year":"2011"},{"issue":"3","key":"key2020100208591575700_ref007","first-page":"27","article-title":"LIBSVM: a library for support vector machines","volume":"2","year":"2011","journal-title":"ACM Transactions on Intelligent Systems and Technology (TIST)"},{"issue":"3","key":"key2020100208591575700_ref008","doi-asserted-by":"crossref","first-page":"478","DOI":"10.1108\/IntR-08-2016-0243","article-title":"Five-star or thumbs-up? The influence of rating system types on users\u2019 perceptions of information quality, cognitive effort, enjoyment and continuance intention","volume":"27","year":"2017","journal-title":"Internet Research"},{"key":"key2020100208591575700_ref010","first-page":"116","article-title":"Battling the internet water army: detection of hidden paid posters","year":"2013"},{"key":"key2020100208591575700_ref011","first-page":"71","article-title":"Detecting offensive language in social media to protect adolescent online safety","year":"2012"},{"issue":"5","key":"key2020100208591575700_ref009","doi-asserted-by":"crossref","first-page":"1039","DOI":"10.1108\/IntR-05-2016-0140","article-title":"The joint effect of association-based corporate posting strategy and eWOM comment valence on social media","volume":"27","year":"2017","journal-title":"Internet Research"},{"key":"key2020100208591575700_ref012","first-page":"1217","article-title":"Anyone can become a troll: causes of trolling behavior in online discussions","year":"2017"},{"issue":"3","key":"key2020100208591575700_ref013","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1108\/IntR-07-2016-0198","article-title":"E-WOM messaging on social media: social ties, temporal distance, and message concreteness","volume":"27","year":"2017","journal-title":"Internet Research"},{"issue":"2","key":"key2020100208591575700_ref014","doi-asserted-by":"crossref","first-page":"356","DOI":"10.1080\/14680777.2015.1008750","article-title":"\u2018It\u2019s Like She\u2019s Eager to be Verbally Abused\u2019: Twitter, trolls, and (en)gendering disciplinary rhetoric","volume":"15","year":"2015","journal-title":"Feminist Media Studies"},{"key":"key2020100208591575700_ref015","volume-title":"Text Processing with GATE","year":"2011"},{"key":"key2020100208591575700_ref016","first-page":"91","article-title":"Seminar users in the Arabic Twitter sphere","year":"2017"},{"issue":"10","key":"key2020100208591575700_ref017","doi-asserted-by":"crossref","first-page":"1577","DOI":"10.1287\/mnsc.1060.0567","article-title":"Strategic manipulation of internet opinion forums: implications for consumers and firms","volume":"52","year":"2006","journal-title":"Management Science"},{"key":"key2020100208591575700_ref018","first-page":"138","article-title":"Automatic evaluation of machine translation quality using n-gram co-occurrence statistics","year":"2002"},{"key":"key2020100208591575700_ref019","unstructured":"Donath, J. (1999), \u201cIdentity and deception in the virtual community\u201d, in Smith, M. and Kollock, P. (Eds), Communities in Cyberspace, Routledge, London and New York, NY, pp. 29-59."},{"issue":"1","key":"key2020100208591575700_ref020","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1108\/IntR-10-2014-0244","article-title":"E-retailing ethics and its impact on customer satisfaction and repurchase intention: a cultural and commitment-trust theory perspective","volume":"26","year":"2016","journal-title":"Internet Research"},{"key":"key2020100208591575700_ref021","first-page":"419","article-title":"Supervised machine learning for the detection of troll profiles in Twitter social network: application to a real case of cyberbullying","year":"2014"},{"key":"key2020100208591575700_ref022","first-page":"27","article-title":"Seven words you can\u2019t say on answerbag: contested terms and conflict in a social Q&A community","year":"2016"},{"key":"key2020100208591575700_ref023","article-title":"A context-aware approach for detecting worth-checking claims in political debates","year":"2017"},{"key":"key2020100208591575700_ref024","first-page":"113","article-title":"Feature-rich named entity recognition for Bulgarian using conditional random fields","year":"2009"},{"key":"key2020100208591575700_ref025","first-page":"805","article-title":"Pairwise neural machine translation evaluation","year":"2015"},{"issue":"C","key":"key2020100208591575700_ref026","first-page":"180","article-title":"Machine translation evaluation with neural networks","volume":"45","year":"2017","journal-title":"Computer Speech & Language"},{"issue":"5","key":"key2020100208591575700_ref027","doi-asserted-by":"crossref","first-page":"1030","DOI":"10.1108\/IntR-08-2014-0201","article-title":"Antecedents of attitudes toward eWOM communication: differences across channels","volume":"26","year":"2016","journal-title":"Internet Research"},{"issue":"1","key":"key2020100208591575700_ref028","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1108\/IntR-03-2014-0084","article-title":"Temporal effects of information from social networks on online behavior: the role of cognitive and affective trust","volume":"26","year":"2016","journal-title":"Internet Research"},{"key":"key2020100208591575700_ref029","first-page":"172","article-title":"In search of credible news","year":"2016"},{"key":"key2020100208591575700_ref030","first-page":"1835","article-title":"Detecting check-worthy factual claims in presidential debates","year":"2015"},{"issue":"5","key":"key2020100208591575700_ref031","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1080\/01972240290108186","article-title":"Searching for safety online: managing \u2018trolling\u2019 in a feminist forum","volume":"18","year":"2002","journal-title":"The Information Society"},{"issue":"1","key":"key2020100208591575700_ref032","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1108\/10662241211199942","article-title":"How social identification and trust influence organizational online knowledge sharing","volume":"22","year":"2012","journal-title":"Internet Research"},{"issue":"3","key":"key2020100208591575700_ref033","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1108\/IntR-01-2013-0007","article-title":"Understanding online shopping intention: the roles of four types of trust and their antecedents","volume":"24","year":"2014","journal-title":"Internet Research"},{"key":"key2020100208591575700_ref034","first-page":"168","article-title":"Mining and summarizing customer reviews","year":"2004"},{"issue":"5","key":"key2020100208591575700_ref035","doi-asserted-by":"crossref","first-page":"591","DOI":"10.1108\/10662241211271563","article-title":"The impact of electronic word of mouth on a tourism destination choice: testing the theory of planned behavior (TBP)","volume":"22","year":"2012","journal-title":"Internet Research"},{"key":"key2020100208591575700_ref036","article-title":"ClaimRank: detecting check-worthy claims in Arabic and English","year":"2018"},{"key":"key2020100208591575700_ref037","first-page":"573","article-title":"Global thread-level inference for comment classification in community question answering","year":"2015"},{"key":"key2020100208591575700_ref038","first-page":"703","article-title":"Joint learning with global inference for comment classification in community question answering","year":"2016"},{"key":"key2020100208591575700_ref039","first-page":"249","article-title":"Sentiment analysis in Twitter for Macedonian","year":"2015"},{"key":"key2020100208591575700_ref040","first-page":"1557","article-title":"On the impact of seed words on sentiment polarity lexicon induction","year":"2016"},{"key":"key2020100208591575700_ref041","first-page":"266","article-title":"Fine-grained sentiment analysis for movie reviews in Bulgarian","year":"2015"},{"key":"key2020100208591575700_ref042","first-page":"334","article-title":"We built a fake news & click-bait filter: what happened next will blow your mind!","year":"2017"},{"key":"key2020100208591575700_ref043","first-page":"344","article-title":"Fully automated fact checking using external sources","year":"2017"},{"issue":"5","key":"key2020100208591575700_ref044","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1108\/10662241211271554","article-title":"Beyond price, how does trust encourage online group\u2019s buying intention?","volume":"22","year":"2012","journal-title":"Internet Research"},{"key":"key2020100208591575700_ref045","first-page":"188","article-title":"Accurately detecting trolls in Slashdot Zoo via decluttering","year":"2014"},{"key":"key2020100208591575700_ref046","first-page":"857","article-title":"An army of me: sockpuppets in online discussion communities","year":"2017"},{"issue":"4","key":"key2020100208591575700_ref047","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1108\/IntR-02-2017-0072","article-title":"Is offensive commenting contagious online? Examining public vs interpersonal swearing in response to Donald Trump\u2019s YouTube campaign videos","volume":"27","year":"2017","journal-title":"Internet Research"},{"issue":"2-3","key":"key2020100208591575700_ref048","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1007\/s10590-009-9059-4","article-title":"The METEOR metric for automatic evaluation of machine translation","volume":"23","year":"2009","journal-title":"Machine Translation"},{"issue":"6380","key":"key2020100208591575700_ref049","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1126\/science.aao2998","article-title":"The science of fake news","volume":"359","year":"2018","journal-title":"Science"},{"issue":"5","key":"key2020100208591575700_ref050","doi-asserted-by":"crossref","first-page":"1112","DOI":"10.1108\/IntR-03-2015-0063","article-title":"Do buyers express their true assessment? Antecedents and consequences of customer praise feedback behaviour on Taobao","volume":"26","year":"2016","journal-title":"Internet Research"},{"issue":"2","key":"key2020100208591575700_ref051","doi-asserted-by":"crossref","first-page":"362","DOI":"10.1108\/IntR-01-2016-0017","article-title":"Effectiveness of online consumer reviews: the influence of valence, reviewer ethnicity, social distance and source trustworthiness","volume":"27","year":"2017","journal-title":"Internet Research"},{"key":"key2020100208591575700_ref052","first-page":"708","article-title":"Detect rumors in microblog posts using propagation structure via kernel learning","year":"2017"},{"key":"key2020100208591575700_ref053","first-page":"3818","article-title":"Detecting rumors from microblogs with recurrent neural networks","year":"2016"},{"key":"key2020100208591575700_ref054","first-page":"399","article-title":"Hunting for troll comments in news community forums","year":"2016"},{"key":"key2020100208591575700_ref055","first-page":"879","article-title":"SemanticZ at SemEval-2016 Task 3: ranking relevant answers in community question answering using semantic similarity based on fine-tuned word embeddings","year":"2016"},{"key":"key2020100208591575700_ref056","first-page":"310","article-title":"Finding opinion manipulation trolls in news community forums","year":"2015"},{"key":"key2020100208591575700_ref057","first-page":"443","article-title":"Exposing paid opinion manipulation trolls","year":"2015"},{"key":"key2020100208591575700_ref058","first-page":"1185","article-title":"Large-scale goodness polarity lexicons for community question answering","year":"2017"},{"key":"key2020100208591575700_ref059","first-page":"879","article-title":"Fact checking in community forums","year":"2018"},{"key":"key2020100208591575700_ref060","first-page":"746","article-title":"Linguistic regularities in continuous space word representations","year":"2013"},{"key":"key2020100208591575700_ref061","first-page":"126","article-title":"A parsimonious language model of social media credibility across disparate events","year":"2017"},{"key":"key2020100208591575700_ref062","first-page":"321","article-title":"NRC-Canada: building the State-of-the-Art in sentiment analysis of tweets","year":"2013"},{"issue":"3","key":"key2020100208591575700_ref063","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1111\/j.1467-8640.2012.00460.x","article-title":"Crowdsourcing a word-emotion association Lexicon","volume":"29","year":"2013","journal-title":"Computational Intelligence"},{"key":"key2020100208591575700_ref064","article-title":"Automatic stance detection using End-to-End memory networks","year":"2018"},{"issue":"3","key":"key2020100208591575700_ref065","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1016\/j.chb.2011.12.005","article-title":"Anonymity and roles associated with aggressive posts in an online forum","volume":"28","year":"2012","journal-title":"Computers in Human Behavior"},{"issue":"1","key":"key2020100208591575700_ref066","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.im.2017.05.001","article-title":"Unveiling the dark side of social networking sites: personal and work-related consequences of social networking site addiction","volume":"55","year":"2018","journal-title":"Information & Management"},{"key":"key2020100208591575700_ref067","first-page":"353","article-title":"Leveraging joint interactions for credibility analysis in news communities","year":"2015"},{"key":"key2020100208591575700_ref068","first-page":"419","article-title":"Building an inflectional stemmer for Bulgarian","year":"2003"},{"key":"key2020100208591575700_ref069","article-title":"BulStem: design and evaluation of inflectional stemmer for Bulgarian","year":"2003"},{"key":"key2020100208591575700_ref070","first-page":"551","article-title":"Do not trust the trolls: predicting credibility in community question answering forums","year":"2017"},{"key":"key2020100208591575700_ref071","first-page":"269","article-title":"SemEval-2015 Task 3: answer selection in community question answering","year":"2015"},{"key":"key2020100208591575700_ref072","first-page":"27","article-title":"SemEval-2017 Task 3: community question answering","year":"2017"},{"key":"key2020100208591575700_ref073","first-page":"525","article-title":"SemEval-2016 Task 3: community question answering","year":"2016"},{"key":"key2020100208591575700_ref074","first-page":"628","article-title":"Automatically identifying good conversations online (yes, they do exist!)","year":"2017"},{"issue":"5","key":"key2020100208591575700_ref075","doi-asserted-by":"crossref","first-page":"1058","DOI":"10.1108\/IntR-09-2016-0276","article-title":"Electronic word of mouth in the Taiwanese social networking community: participation factors","volume":"27","year":"2017","journal-title":"Internet Research"},{"key":"key2020100208591575700_ref076","first-page":"311","article-title":"BLEU: a method for automatic evaluation of machine translation","year":"2002"},{"key":"key2020100208591575700_ref077","first-page":"1003","article-title":"Where the truth lies: explaining the credibility of emerging claims on the web and social media","year":"2017"},{"key":"key2020100208591575700_ref078","first-page":"249","article-title":"Truthy: mapping the spread of Astroturf in microblog streams","year":"2011"},{"issue":"5","key":"key2020100208591575700_ref079","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1108\/14684520710832324","article-title":"A conceptual model of trust in the online environment","volume":"31","year":"2007","journal-title":"Online Information Review"},{"issue":"2","key":"key2020100208591575700_ref080","doi-asserted-by":"crossref","first-page":"677","DOI":"10.1007\/s13042-015-0463-1","article-title":"Content based approach to find the credibility of user in social networks: an application of cyberbullying","volume":"8","year":"2017","journal-title":"International Journal of Machine Learning and Cybernetics"},{"key":"key2020100208591575700_ref081","first-page":"792","article-title":"Troll detection by domain-adapting sentiment analysis","year":"2015"},{"issue":"4","key":"key2020100208591575700_ref082","doi-asserted-by":"crossref","first-page":"674","DOI":"10.1108\/IntR-01-2014-0023","article-title":"Cyber bullying and teachers\u2019 awareness","volume":"25","year":"2015","journal-title":"Internet Research"},{"key":"key2020100208591575700_ref083","first-page":"223","article-title":"A study of translation edit rate with targeted human annotation","year":"2006"},{"key":"key2020100208591575700_ref084","first-page":"455","article-title":"Parsing with compositional vector grammars","year":"2013"},{"key":"key2020100208591575700_ref085","first-page":"590","article-title":"SU-FMI: system description for SemEval-2014 Task 9 on sentiment analysis in Twitter","year":"2014"},{"issue":"6380","key":"key2020100208591575700_ref086","doi-asserted-by":"crossref","first-page":"1146","DOI":"10.1126\/science.aap9559","article-title":"The spread of true and false news online","volume":"359","year":"2018","journal-title":"Science"},{"key":"key2020100208591575700_ref087","first-page":"347","article-title":"Recognizing contextual polarity in phrase-level sentiment analysis","year":"2005"},{"key":"key2020100208591575700_ref088","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1016\/j.chb.2017.10.022","article-title":"Does gender matter in cyberbullying perpetration? An empirical investigation","volume":"79","year":"2018","journal-title":"Computers in Human Behavior"},{"issue":"5","key":"key2020100208591575700_ref089","doi-asserted-by":"crossref","first-page":"1014","DOI":"10.1108\/IntR-07-2016-0223","article-title":"Twitter as customer\u2019s eWOM: an empirical study on their impact on firm financial performance","volume":"27","year":"2017","journal-title":"Internet Research"},{"key":"key2020100208591575700_ref090","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.jbusres.2018.02.016","article-title":"Manufactured opinions: the effect of manipulating online product reviews","volume":"87","year":"2018","journal-title":"Journal of Business Research"}],"container-title":["Internet Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IntR-03-2017-0118\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IntR-03-2017-0118\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T21:35:27Z","timestamp":1753392927000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/intr\/article\/28\/5\/1292-1312\/175901"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,10,2]]},"references-count":90,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2018,10,2]]}},"alternative-id":["10.1108\/IntR-03-2017-0118"],"URL":"https:\/\/doi.org\/10.1108\/intr-03-2017-0118","relation":{},"ISSN":["1066-2243"],"issn-type":[{"value":"1066-2243","type":"print"}],"subject":[],"published":{"date-parts":[[2018,10,2]]}}}