{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,2]],"date-time":"2025-08-02T17:23:50Z","timestamp":1754155430903,"version":"3.41.2"},"reference-count":84,"publisher":"Emerald","issue":"1","license":[{"start":{"date-parts":[[2020,7,21]],"date-time":"2020-07-21T00:00:00Z","timestamp":1595289600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JICES"],"published-print":{"date-parts":[[2021,3,3]]},"abstract":"<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title>\n<jats:p>Over the past decade, many research works in various disciplines have benefited from the endless ocean of people and their potentials (in the form of crowdsourcing) as an effective problem-solving strategy and computational model. But nothing interesting is ever completely one-sided. Therefore, when it comes to leveraging people's power, as the dark side of crowdsourcing, there are some possible threats that have not been considered as should be, such as recruiting black hat crowdworkers for organizing targeted adversarial intentions. The purpose of this paper is to draw more attention to this critical issue through investigation of its different aspects.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title>\n<jats:p>To delve into details of such malicious intentions, the related literature and previous researches have been studied. Then, four major typologies for adversarial crowdsourced attacks as well as some real-world scenarios are discussed and delineated. Finally, possible future threats are introduced.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Findings<\/jats:title>\n<jats:p>Despite many works on adversarial crowdsourcing, there are only a few specific research studies devoted to considering the issue in the context of cyber security. In this regard, the proposed typologies (and addressed scenarios) for such human-mediated attacks can shed light on the way of identifying and confronting such threats.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title>\n<jats:p>To the best of the authors' knowledge, this the first work in which the titular topic is investigated in detail. Due to popularity and efficiency of leveraging crowds' intelligence and efforts in a wide range of application domains, it is most likely that adversarial human-driven intentions gain more attention. In this regard, it is anticipated that the present research study can serve as a roadmap for proposing defensive mechanisms to cope with such diverse threats.<\/jats:p>\n<\/jats:sec>","DOI":"10.1108\/jices-08-2019-0100","type":"journal-article","created":{"date-parts":[[2020,7,21]],"date-time":"2020-07-21T09:18:19Z","timestamp":1595323099000},"page":"87-103","source":"Crossref","is-referenced-by-count":2,"title":["Rogue people: on adversarial crowdsourcing in the context of cyber security"],"prefix":"10.1108","volume":"19","author":[{"given":"Mohammad","family":"Moradi","sequence":"first","affiliation":[]},{"given":"Qi","family":"Li","sequence":"additional","affiliation":[]}],"member":"140","published-online":{"date-parts":[[2020,7,21]]},"reference":[{"key":"key2021043009034652300_ref001","first-page":"44","article-title":"Man-at-the-end attacks: Analysis, taxonomy, human aspects, motivation and future directions","volume":"48","year":"2017","journal-title":"Journal of Network and Computer Applications"},{"issue":"2","key":"key2021043009034652300_ref002","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1109\/MIC.2013.20","article-title":"Quality control in crowdsourcing systems: Issues and directions","volume":"17","year":"2013","journal-title":"IEEE Internet Computing"},{"issue":"1","key":"key2021043009034652300_ref003","first-page":"1","article-title":"Rings: a game with a purpose for test data generation","volume":"30","year":"2017","journal-title":"Interacting with Computers"},{"key":"key2021043009034652300_ref004","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1145\/3176258.3176327","article-title":"Forgetting with puzzles: using cryptographic puzzles to support digital forgetting","volume-title":"Proceedings of the Eighth ACM Conference on Data and Application Security and Privacy","year":"2018"},{"key":"key2021043009034652300_ref005","first-page":"73","article-title":"The computational and storage potential of volunteer computing","volume-title":"Proceedings of Sixth IEEE International Symposium on Cluster Computing and the Grid, CCGRID 06","year":"2006"},{"issue":"6","key":"key2021043009034652300_ref006","doi-asserted-by":"crossref","first-page":"2678","DOI":"10.1016\/j.eswa.2013.11.011","article-title":"Leveraging non-expert crowdsourcing workers for improper task detection in crowdsourcing marketplaces","volume":"41","year":"2014","journal-title":"Expert Systems with Applications"},{"key":"key2021043009034652300_ref007","article-title":"When the customer is wrong: defamation, interactive websites, and immunity","volume":"33","year":"2014","journal-title":"The Review of Litigation"},{"issue":"1","key":"key2021043009034652300_ref008","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1177\/1354856507084420","article-title":"Crowdsourcing as a model for problem solving: an introduction and cases","volume":"14","year":"2008","journal-title":"Convergence: The International Journal of Research into New Media Technologies"},{"issue":"5","key":"key2021043009034652300_ref009","doi-asserted-by":"publisher","DOI":"10.1108\/ITP-12-2017-0422","article-title":"Solutions for counteracting human deception in social engineering attacks","volume":"32","year":"2018","journal-title":"Information Technology and People"},{"year":"2012","key":"key2021043009034652300_ref010","article-title":"Urbanopoly \u2013 a social and location-based game with a purpose to crowdsource your urban data"},{"key":"key2021043009034652300_ref011","first-page":"1322","article-title":"Ka-captcha: an opportunity for knowledge acquisition on the web","volume":"22","year":"2017","journal-title":"Proceedings of the National Conference on Artificial Intelligence, AAAI Press"},{"issue":"1","key":"key2021043009034652300_ref012","article-title":"Quality control in crowdsourcing: a survey of quality attributes, assessment techniques, and assurance actions","volume":"51","year":"2018","journal-title":"ACM Computing Surveys"},{"issue":"5","key":"key2021043009034652300_ref013","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1007\/s12599-019-00595-2","article-title":"Hybrid intelligence","volume":"61","year":"2019","journal-title":"Business and Information Systems Engineering"},{"issue":"10","key":"key2021043009034652300_ref014","article-title":"Why individuals participate in micro-task crowdsourcing work environment: revealing crowdworkers' perceptions","volume":"17","year":"2016","journal-title":"Journal of the Association for Information Systems"},{"first-page":"26","article-title":"Mechanical cheat: spamming schemes and adversarial techniques on crowdsourcing platforms","year":"2012","key":"key2021043009034652300_ref015"},{"first-page":"823","article-title":"Is there PAPA in crowd work? A literature review on ethical dimensions in crowdsourcing","year":"2016","key":"key2021043009034652300_ref016"},{"key":"key2021043009034652300_ref017","first-page":"28","article-title":"Location-based crowdsourcing in disaster response","volume-title":"Proceedings of14th International Conference on Advances in Mobile Computing and Multi Media","year":"2016"},{"issue":"1","key":"key2021043009034652300_ref018","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1109\/MCOM.2014.6710071","article-title":"Smartphone-based crowdsourcing for network monitoring: opportunities, challenges, and a case study","volume":"52","year":"2014","journal-title":"IEEE Communications Magazine"},{"key":"key2021043009034652300_ref019","first-page":"10","article-title":"A decision tree based quality control framework for multi-phase tasks in crowdsourcing","volume-title":"Proceedings of 12th Chinese Conference on Computer Supported Cooperative Work and Social Computing","year":"2017"},{"key":"key2021043009034652300_ref020","first-page":"46","article-title":"CAPTCHA-based image labeling on the soylent grid","volume-title":"Proceedings of ACM SIGKDD Workshop on Human Computation","year":"2016"},{"issue":"8","key":"key2021043009034652300_ref021","doi-asserted-by":"crossref","first-page":"1199","DOI":"10.1002\/sec.825","article-title":"The design and implementation of a secure CAPTCHA against man\u2010in\u2010the\u2010middle attacks","volume":"7","year":"2014","journal-title":"Security and Communication Networks"},{"key":"key2021043009034652300_ref022","first-page":"1631","article-title":"Understanding malicious behavior in crowdsourcing platforms: the case of online surveys","volume-title":"Proceedings of 33rd Annual ACM Conference on Human Factors in Computing Systems","year":"2015"},{"key":"key2021043009034652300_ref023","first-page":"33","article-title":"Breaking bad: understanding behavior of crowd workers in categorization microtasks","volume-title":"Proceedings of 26th ACM Conference on Hypertext and Social Media","year":"2015"},{"issue":"11","key":"key2021043009034652300_ref024","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/MCOM.2011.6069707","article-title":"Mobile crowdsensing: current state and future challenges","volume":"49","year":"2011","journal-title":"IEEE Communications Magazine"},{"key":"key2021043009034652300_ref025","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1007\/978-3-642-39498-0_13","article-title":"Analysis of ecrime in crowd-sourced labor markets: mechanical turk vs freelancer","volume-title":"The Economics of Information Security and Privacy","year":"2013"},{"first-page":"176","article-title":"Tracking the evolution of communities in dynamic social networks","year":"2010","key":"key2021043009034652300_ref026"},{"first-page":"1314","article-title":"Dirty deeds done dirt cheap: a darker side to crowdsourcing","year":"2011","key":"key2021043009034652300_ref027"},{"first-page":"2542","article-title":"High quality participant recruitment in vehicle-based crowdsourcing using predictable mobility","year":"2015","key":"key2021043009034652300_ref028"},{"year":"2014","key":"key2021043009034652300_ref029","article-title":"Hacker fakes German minister's fingerprints using photos of her hands"},{"first-page":"316","article-title":"Cost-optimal validation mechanisms and cheat-detection for crowdsourcing platforms","year":"2011","key":"key2021043009034652300_ref030"},{"key":"key2021043009034652300_ref031","first-page":"1","article-title":"The rise of crowdsourcing","volume":"14","year":"2006","journal-title":"Wired Magazine"},{"first-page":"1137","article-title":"Anti-malicious crowdsourcing using the zero-determinant strategy","year":"2017","key":"key2021043009034652300_ref032"},{"issue":"6","key":"key2021043009034652300_ref033","doi-asserted-by":"crossref","first-page":"855","DOI":"10.1007\/s00778-017-0484-3","article-title":"Answer validation for generic crowdsourcing tasks with minimal efforts","volume":"26","year":"2017","journal-title":"The VLDB Journal"},{"key":"key2021043009034652300_ref034","first-page":"2492","article-title":"Reputation-based worker filtering in crowdsourcing","year":"2014","journal-title":"Advances in Neural Information Processing Systems"},{"key":"key2021043009034652300_ref035","first-page":"96","article-title":"Instance-privacy preserving crowdsourcing","volume-title":"Second AAAI Conference on Human Computation and Crowdsourcing","year":"2014"},{"first-page":"37","article-title":"Privacy attitudes of mechanical Turk workers and the us public","year":"2014","key":"key2021043009034652300_ref036"},{"first-page":"71","article-title":"Putting out a HIT: crowdsourcing malware installs","year":"2011","key":"key2021043009034652300_ref037"},{"key":"key2021043009034652300_ref038","first-page":"2254","article-title":"User recruitment for mobile crowdsensing over opportunistic networks","volume-title":"Proceedings of IEEE Conference on computer Communications (INFOCOM)","year":"2015"},{"year":"2017","key":"key2021043009034652300_ref039","article-title":"Crowdmask: using crowds to preserve privacy in crowd-powered systems via progressive filtering"},{"key":"key2021043009034652300_ref040","first-page":"1941","article-title":"Worker types and personality traits in crowdsourcing relevance labels","volume-title":"Proceedings of the 20th ACM International Conferenceon Information and Knowledge Management","year":"2011"},{"key":"key2021043009034652300_ref041","first-page":"2583","article-title":"The face of quality in crowdsourcing relevance labels: demographics, personality and labeling accuracy","volume-title":"Proceedings of 21st ACM International Conference on Information and Knowledge Management","year":"2012"},{"article-title":"Detecting non-adversarial collusion in crowdsourcing","volume-title":"Second AAAI Conference on Human Computation and Crowdsourcing (HCOMP)","year":"2014","key":"key2021043009034652300_ref042"},{"issue":"2","key":"key2021043009034652300_ref043","doi-asserted-by":"crossref","first-page":"1215","DOI":"10.1007\/s11042-013-1422-z","article-title":"FaceCAPTCHA: a CAPTCHA that identifies the gender of face images unrecognized by existing gender classifiers","volume":"72","year":"2014","journal-title":"Multimedia Tools and Applications"},{"year":"2015","key":"key2021043009034652300_ref044","article-title":"The cost of asking crowd workers to behave maliciously"},{"first-page":"267","article-title":"Crowdsourcing and mining crowd data","year":"2015","key":"key2021043009034652300_ref045"},{"first-page":"331","article-title":"Crowdturfers, campaigns, and social media: tracking and revealing crowdsourced manipulation of social media","year":"2013","key":"key2021043009034652300_ref046"},{"key":"key2021043009034652300_ref047","first-page":"1","article-title":"Mechanism design games for thwarting malicious behavior in crowdsourcing applications","volume-title":"Proceedings of IEEE Conference on Computer Communications, INFOCOM","year":"2017"},{"key":"key2021043009034652300_ref048","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.ijhcs.2016.09.008","article-title":"Situated crowdsourcing during disasters: managing the tasks of spontaneous volunteers through public displays","volume":"102","year":"2017","journal-title":"International Journal of Human-Computer Studies"},{"issue":"9","key":"key2021043009034652300_ref049","doi-asserted-by":"crossref","first-page":"2234","DOI":"10.1109\/TMC.2015.2485978","article-title":"Incentive mechanism design for heterogeneous crowdsourcing using all-pay contests","volume":"15","year":"2016","journal-title":"IEEE Transactions on Mobile Computing"},{"key":"key2021043009034652300_ref050","doi-asserted-by":"crossref","first-page":"1419","DOI":"10.1016\/j.respol.2015.05.010","article-title":"Solver engagement in knowledge sharing in crowdsourcing communities: exploring the link to creativity","volume":"44","year":"2015","journal-title":"Research Policy"},{"key":"key2021043009034652300_ref051","first-page":"13","article-title":"Attack under disguise: an intelligent data poisoning attack mechanism in crowdsourcing","volume-title":"Proceedings of 2018 World Wide Web Conference on World Wide Web, International World Wide Web Conferences Steering Committee","year":"2018"},{"key":"key2021043009034652300_ref052","first-page":"111","article-title":"Towards data poisoning attacks in crowd sensing systems","volume-title":"Proceedings of18th ACM International Symposium on Mobile Ad Hoc Networking and Computing","year":"2018"},{"issue":"2","key":"key2021043009034652300_ref053","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1108\/IJCS-03-2019-0012","article-title":"Collective hybrid intelligence: towards a conceptual framework","volume":"3","year":"2019","journal-title":"International Journal of Crowd Science"},{"key":"key2021043009034652300_ref055","first-page":"71","article-title":"An analysis of underground forums","volume-title":"Proceedings of ACM SIGCOMM Conference on Internet Measurement Conference","year":"2011"},{"year":"2010","key":"key2021043009034652300_ref054","article-title":"Re: CAPTCHAs-understanding CAPTCHA-solving services in an economic context"},{"first-page":"1","article-title":"Working on low-paid micro-task crowdsourcing platforms: an existence, relatedness and growth view","year":"2015","key":"key2021043009034652300_ref056"},{"year":"2015","key":"key2021043009034652300_ref057","article-title":"Participant engagement in community crowdsourcing ECIS"},{"key":"key2021043009034652300_ref058","first-page":"3966","article-title":"Disruption and deception in crowdsourcing: towards a crowdsourcing risk framework","volume-title":"Proceedings of the 51st HI International Conference on System Sciences","year":"2018"},{"first-page":"257","article-title":"Crowdsourcing attacks on biometric systems","year":"2014","key":"key2021043009034652300_ref059"},{"key":"key2021043009034652300_ref060","article-title":"Mad men posing as ordinary consumers: the essential role of self-regulation and industry ethics on decreasing deceptive online consumer ratings and reviews","volume":"12","year":"2012","journal-title":"The John Marshall Review of Intellectual Property Law"},{"key":"key2021043009034652300_ref061","first-page":"59","article-title":"Protecting brand image or gaming the system: consumer gag contracts in an age of crowdsourced ratings and reviews","volume":"7","year":"2016","journal-title":"William and Mary Business Law Review"},{"key":"key2021043009034652300_ref062","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2017.08.025","article-title":"Cecoin: a decentralized PKI mitigating MitM attacks","volume":"107","year":"2017","journal-title":"Future Generation Computer Systems"},{"first-page":"31","article-title":"Exploring hacker assets in underground forums","year":"2015","key":"key2021043009034652300_ref063"},{"first-page":"243","article-title":"Booters \u2013 an analysis of DDoS-as-a-service attacks","year":"2015","key":"key2021043009034652300_ref064"},{"first-page":"364","article-title":"A systematic literature review of crowdsourcing-based research in information security","year":"2016","key":"key2021043009034652300_ref065"},{"first-page":"531","article-title":"The good, the bad and the ugly: why crowdsourcing needs ethics","year":"2013","key":"key2021043009034652300_ref066"},{"key":"key2021043009034652300_ref067","doi-asserted-by":"crossref","first-page":"758","DOI":"10.1016\/j.future.2017.12.039","article-title":"Cooperative trust relaying and privacy preservation via edge-crowdsourcing in social internet of things","volume":"92","year":"2019","journal-title":"Future Generation Computer Systems"},{"first-page":"4439","article-title":"Avoiding imposters and delinquents: adversarial crowdsourcing and peer prediction","year":"2016","key":"key2021043009034652300_ref068"},{"issue":"4","key":"key2021043009034652300_ref069","doi-asserted-by":"crossref","first-page":"2201","DOI":"10.1109\/TIT.2017.2659660","article-title":"Adversarial top-$K $ranking","volume":"63","year":"2017","journal-title":"IEEE Transactions on Information Theory"},{"issue":"3","key":"key2021043009034652300_ref070","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1108\/09593841211254358","article-title":"Analyzing trajectories of information security awareness","volume":"25","year":"2012","journal-title":"Information Technology and People"},{"issue":"6","key":"key2021043009034652300_ref071","first-page":"92","article-title":"Games with a purpose","volume":"39","year":"2008","journal-title":"Computer"},{"first-page":"21","article-title":"How much spam can you take? An analysis of crowdsourcing results to increase accuracy","year":"2011","key":"key2021043009034652300_ref072"},{"first-page":"65","article-title":"A participant recruitment framework for crowdsourcing based software requirement acquisition","year":"2014","key":"key2021043009034652300_ref073"},{"year":"2013","key":"key2021043009034652300_ref074","article-title":"Quality-based pricing for crowdsourced workersTechnical report"},{"key":"key2021043009034652300_ref075","first-page":"392","article-title":"GeoCAPTCHA \u2013 a novel personalized CAPTCHA using geographic concept to defend against 3rd party human attack","volume-title":"Proceedings of IEEE 31st International Performance Computing and Communications Conference (IPCCC)","year":"2012"},{"issue":"2","key":"key2021043009034652300_ref076","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1016\/j.bushor.2016.11.009","article-title":"Crowdsourcing in a time of empowered stakeholders: lessons from crowdsourcing campaigns","volume":"60","year":"2017","journal-title":"Business Horizons"},{"key":"key2021043009034652300_ref077","article-title":"Our privacy needs to be protected at all costs: crowd workers' privacy experiences on amazon mechanical turk","volume":"1","year":"2017","journal-title":"Proceedings of the ACM on Human-Computer Interaction"},{"issue":"1","key":"key2021043009034652300_ref078","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1108\/17440080680000099","article-title":"Protecting web applications from DDoS attacks by an active distributed defense system","volume":"2","year":"2006","journal-title":"International Journal of Web Information Systems"},{"key":"key2021043009034652300_ref079","first-page":"2533","article-title":"Revealing, characterizing, and detecting crowdsourcing spammers: a case study in community Q&A","volume-title":"Proceedings of IEEE Conference on Computer Communications (INFOCOM)","year":"2015"},{"issue":"3","key":"key2021043009034652300_ref080","doi-asserted-by":"crossref","first-page":"1732","DOI":"10.1109\/TNET.2015.2421897","article-title":"Incentive mechanisms for crowdsensing: crowdsourcing with smartphones","volume":"24","year":"2016","journal-title":"IEEE\/ACM Transactions on Networking (Networking)"},{"key":"key2021043009034652300_ref081","first-page":"1","article-title":"Crowdrec: trust-aware worker recommendation in crowdsourcing environments","volume-title":"Proceedings of IEEE International Conference on Web Services (ICWS)","year":"2016"},{"key":"key2021043009034652300_ref082","first-page":"245","article-title":"CrowdDefense: a trust vector-based threat defense model in crowdsourcing environments","volume-title":"Proceedings of IEEE International Conference on Web Services (ICWS)","year":"2017"},{"issue":"15","key":"key2021043009034652300_ref083","doi-asserted-by":"crossref","first-page":"2179","DOI":"10.1002\/wcm.2675","article-title":"CAPR: context\u2010aware participant recruitment mechanism in mobile crowdsourcing","volume":"16","year":"2016","journal-title":"Wireless Communications and Mobile Computing"},{"key":"key2021043009034652300_ref084","first-page":"2830","article-title":"Truthful incentive mechanisms for crowdsourcing","volume-title":"Proceedings of IEEE Conference on Computer Communications (INFOCOM)","year":"2015"}],"container-title":["Journal of Information, Communication and Ethics in Society"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/JICES-08-2019-0100\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/JICES-08-2019-0100\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T21:38:39Z","timestamp":1753393119000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/jices\/article\/19\/1\/87-103\/213946"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,21]]},"references-count":84,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,7,21]]},"published-print":{"date-parts":[[2021,3,3]]}},"alternative-id":["10.1108\/JICES-08-2019-0100"],"URL":"https:\/\/doi.org\/10.1108\/jices-08-2019-0100","relation":{},"ISSN":["1477-996X","1477-996X"],"issn-type":[{"type":"print","value":"1477-996X"},{"type":"print","value":"1477-996X"}],"subject":[],"published":{"date-parts":[[2020,7,21]]}}}