{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T18:52:01Z","timestamp":1772823121757,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":63,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,9,16]],"date-time":"2024-09-16T00:00:00Z","timestamp":1726444800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,9,16]]},"DOI":"10.1145\/3703465.3703472","type":"proceedings-article","created":{"date-parts":[[2025,1,16]],"date-time":"2025-01-16T12:18:31Z","timestamp":1737029911000},"page":"84-95","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Is Crowdsourcing a Puppet Show? Detecting a New Type of Fraud in Online Platforms"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-2423-7305","authenticated-orcid":false,"given":"Shengqian","family":"Wang","sequence":"first","affiliation":[{"name":"Ontario Tech University, Oshawa, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3644-3837","authenticated-orcid":false,"given":"Israt Jahan","family":"Jui","sequence":"additional","affiliation":[{"name":"Ontario Tech University, Oshawa, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6629-158X","authenticated-orcid":false,"given":"Julie","family":"Thorpe","sequence":"additional","affiliation":[{"name":"Ontario Tech University, Oshawa, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,1,16]]},"reference":[{"key":"e_1_3_3_1_2_2","doi-asserted-by":"crossref","unstructured":"Herman Aguinis Isabel Villamor and Ravi\u00a0S Ramani. 2021. MTurk research: Review and recommendations. Journal of Management 47 4 (2021) 823\u2013837.","DOI":"10.1177\/0149206320969787"},{"key":"e_1_3_3_1_3_2","unstructured":"Hui Bai. 2018. Evidence that a large amount of low quality responses on MTurk can be detected with repeated GPS coordinates. https:\/\/www.maxhuibai.com\/blog\/evidence-that-responses-from-repeating-gps-are-random. Online; Accessed: 2023-09-14."},{"key":"e_1_3_3_1_4_2","doi-asserted-by":"crossref","unstructured":"Adam\u00a0J Berinsky Michele\u00a0F Margolis and Michael\u00a0W Sances. 2014. Separating the shirkers from the workers? Making sure respondents pay attention on self-administered surveys. American journal of political science 58 3 (2014) 739\u2013753.","DOI":"10.1111\/ajps.12081"},{"key":"e_1_3_3_1_5_2","volume-title":"Social science research: Principles, methods, and practices","author":"Bhattacherjee Anol","year":"2012","unstructured":"Anol Bhattacherjee. 2012. Social science research: Principles, methods, and practices. University of South Florida."},{"key":"e_1_3_3_1_6_2","unstructured":"John Bohannon. 2016. Psychologists grow increasingly dependent on online research subjects. https:\/\/www.science.org\/content\/article\/psychologists-grow-increasingly-dependent-online-research-subjects\/."},{"key":"e_1_3_3_1_7_2","doi-asserted-by":"crossref","unstructured":"Florian Br\u00fchlmann Serge Petralito Lena\u00a0F. Aeschbach and Klaus Opwis. 2020. The quality of data collected online: An investigation of careless responding in a crowdsourced sample. Methods in Psychology 2 (2020) 100022.","DOI":"10.1016\/j.metip.2020.100022"},{"key":"e_1_3_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2011-764"},{"key":"e_1_3_3_1_9_2","doi-asserted-by":"crossref","unstructured":"Jesse Chandler Cheskie Rosenzweig Aaron\u00a0J Moss Jonathan Robinson and Leib Litman. 2019. Online panels in social science research: Expanding sampling methods beyond Mechanical Turk. Behavior research methods 51 (2019) 2022\u20132038.","DOI":"10.3758\/s13428-019-01273-7"},{"key":"e_1_3_3_1_10_2","doi-asserted-by":"crossref","unstructured":"Jesse Chandler and Danielle Shapiro. 2016. Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology 12 (2016) 53\u201381.","DOI":"10.1146\/annurev-clinpsy-021815-093623"},{"key":"e_1_3_3_1_11_2","doi-asserted-by":"crossref","unstructured":"Jesse Chandler Itay Sisso and Danielle Shapiro. 2020. Participant carelessness and fraud: Consequences for clinical research and potential solutions. Journal of abnormal psychology 129 (2020) 49\u201355.","DOI":"10.1037\/abn0000479"},{"key":"e_1_3_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/1517494.1517498"},{"key":"e_1_3_3_1_13_2","doi-asserted-by":"crossref","unstructured":"Michael Chmielewski and Sarah\u00a0C Kucker. 2020. An MTurk crisis? Shifts in data quality and the impact on study results. Social Psychological and Personality Science 11 4 (2020) 464\u2013473.","DOI":"10.1177\/1948550619875149"},{"key":"e_1_3_3_1_14_2","doi-asserted-by":"crossref","unstructured":"Marvin\u00a0M Chun and Yuhong Jiang. 1998. Contextual cueing: Implicit learning and memory of visual context guides spatial attention. Cognitive psychology 36 1 (1998) 28\u201371.","DOI":"10.1006\/cogp.1998.0681"},{"key":"e_1_3_3_1_15_2","unstructured":"CloudResearch. 2024. CloudResearch. https:\/\/www.cloudresearch.com\/. Online; Accessed: 2024-04-01."},{"key":"e_1_3_3_1_16_2","doi-asserted-by":"crossref","unstructured":"Sean\u00a0A. Dennis Brian\u00a0M. Goodson and Chris Pearson. 2019. Online worker fraud and evolving threats to the integrity of MTurk Data: A discussion of virtual private servers and the limitations of IP-Based screening procedures. Research Methods & Methodology in Accounting eJournal 32 1 (2019) 119\u2013134.","DOI":"10.2308\/bria-18-044"},{"key":"e_1_3_3_1_17_2","doi-asserted-by":"crossref","unstructured":"Sean\u00a0A. Dennis Brian\u00a0M. Goodson and Christopher\u00a0A. Pearson. 2020. Online worker fraud and evolving threats to the integrity of MTurk Data: A discussion of virtual private servers and the limitations of IP-Based screening procedures. Behavioral Research in Accounting 32 1 (2020) 119\u2013134.","DOI":"10.2308\/bria-18-044"},{"key":"e_1_3_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159661"},{"key":"e_1_3_3_1_19_2","unstructured":"Djellel\u00a0Eddine Difallah Gianluca Demartini and Philippe Cudr\u00e9-Mauroux. 2012. Mechanical cheat: Spamming schemes and adversarial techniques on crowdsourcing platforms. CrowdSearch 842 (2012) 26\u201330."},{"key":"e_1_3_3_1_20_2","doi-asserted-by":"crossref","unstructured":"Benjamin\u00a0D Douglas Patrick\u00a0J Ewell and Markus Brauer. 2023. Data quality in online human-subjects research: Comparisons between MTurk Prolific CloudResearch Qualtrics and SONA. Plos one 18 3 (2023) e0279720.","DOI":"10.1371\/journal.pone.0279720"},{"key":"e_1_3_3_1_21_2","unstructured":"Emily Dreyfuss. 2018. A Bot Panic Hits Amazon\u2019s Mechanical Turk. https:\/\/www.wired.com\/story\/amazon-mechanical-turk-bot-panic\/. Online; Accessed: 2023-09-14."},{"key":"e_1_3_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3537674.3555783"},{"key":"e_1_3_3_1_23_2","doi-asserted-by":"crossref","unstructured":"Carsten Eickhoff and Arjen\u00a0P de Vries. 2013. Increasing cheat robustness of crowdsourcing tasks. Information retrieval 16 (2013) 121\u2013137.","DOI":"10.1007\/s10791-011-9181-9"},{"key":"e_1_3_3_1_24_2","unstructured":"EUREC. 2024. The European Network of Research Ethics Committees (EUREC). https:\/\/www.enrio.eu\/the-european-network-of-reserch-ethics-committees-eurec\/. Online; Accessed: 2024-04-01."},{"key":"e_1_3_3_1_25_2","doi-asserted-by":"crossref","unstructured":"Daniele Fanelli. 2009. How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PloS one 4 5 (2009) e5738.","DOI":"10.1371\/journal.pone.0005738"},{"key":"e_1_3_3_1_26_2","unstructured":"FingerprintJS. 2023. FingerprintJS Github. https:\/\/github.com\/fingerprintjs\/fingerprintjs\/."},{"key":"e_1_3_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702443"},{"key":"e_1_3_3_1_28_2","unstructured":"The government\u00a0of Canada. 2024. Criminal Code (R.S.C. 1985 c. C-46). https:\/\/laws-lois.justice.gc.ca\/eng\/acts\/c-46\/section-403.html\/. Online; Accessed: 2024-04-26."},{"key":"e_1_3_3_1_29_2","doi-asserted-by":"crossref","unstructured":"Christine Grady. 2015. Institutional review boards: Purpose and challenges. Chest 148 5 (2015) 1148\u20131155.","DOI":"10.1378\/chest.15-0706"},{"key":"e_1_3_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781351137713-17"},{"key":"e_1_3_3_1_31_2","doi-asserted-by":"crossref","unstructured":"David\u00a0J Hauser Aaron\u00a0J Moss Cheskie Rosenzweig Shalom\u00a0N Jaffe Jonathan Robinson and Leib Litman. 2022. Evaluating CloudResearch\u2019s approved group as a solution for problematic data quality on MTurk. Behavior research methods 55 (2022) 3953\u20133964.","DOI":"10.3758\/s13428-022-01999-x"},{"key":"e_1_3_3_1_32_2","doi-asserted-by":"crossref","unstructured":"David\u00a0J Hauser and Norbert Schwarz. 2016. Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior research methods 48 (2016) 400\u2013407.","DOI":"10.3758\/s13428-015-0578-z"},{"key":"e_1_3_3_1_33_2","doi-asserted-by":"crossref","unstructured":"Michael Hechter and Satoshi Kanazawa. 1997. Sociological rational choice theory. Annual review of sociology 23 1 (1997) 191\u2013214.","DOI":"10.1146\/annurev.soc.23.1.191"},{"key":"e_1_3_3_1_34_2","doi-asserted-by":"crossref","unstructured":"Nicholas\u00a0C Hunt and Andrea\u00a0M Scheetz. 2019. Using MTurk to distribute a survey or experiment: Methodological considerations. Journal of Information Systems 33 1 (2019) 43\u201365.","DOI":"10.2308\/isys-52021"},{"key":"e_1_3_3_1_35_2","doi-asserted-by":"crossref","unstructured":"Ah\u00a0Reum Kang Huy\u00a0Kang Kim and Jiyoung Woo. 2012. Chatting pattern based game BOT detection: do they talk like us? KSII Transactions on Internet and Information Systems (TIIS) 6 11 (2012) 2866\u20132879.","DOI":"10.3837\/tiis.2012.11.007"},{"key":"e_1_3_3_1_36_2","volume-title":"Assessing the risks to online polls from bogus respondents","author":"Kennedy Courtney","year":"2020","unstructured":"Courtney Kennedy, Nick Hatley, Arnold Lau, Andrew Mercer, Scott Keeter, Joshua Ferno, and Dorene Asare-Marfo. 2020. Assessing the risks to online polls from bogus respondents. Pew Research Center."},{"key":"e_1_3_3_1_37_2","doi-asserted-by":"crossref","unstructured":"Ryan Kennedy Scott Clifford Tyler Burleigh Philip\u00a0D. Waggoner Ryan Jewell and Nicholas J.\u00a0G. Winter. 2020. The shape of and solutions to the MTurk quality crisis. Political Science Research and Methods 8 4 (2020) 614\u2013629.","DOI":"10.1017\/psrm.2020.6"},{"key":"e_1_3_3_1_38_2","unstructured":"David Kushner. 2010. Steamed: Valve software battles video-game cheaters. IEEE Spectrum 10 (2010) 2010."},{"key":"e_1_3_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2016.23436"},{"key":"e_1_3_3_1_40_2","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1109\/SMC.2016.7844243","volume-title":"2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC)","author":"Ma Lei","year":"2016","unstructured":"Lei Ma, Chungang Yan, Peihai Zhao, and Mimi Wang. 2016. A kind of mouse behavior authentication method on dynamic soft keyboard. In 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 211\u2013216."},{"key":"e_1_3_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3578503.3583622"},{"key":"e_1_3_3_1_42_2","doi-asserted-by":"crossref","unstructured":"Adam\u00a0W Meade and S\u00a0Bartholomew Craig. 2012. Identifying careless responses in survey data. Psychological methods 17 3 (2012) 437.","DOI":"10.1037\/a0028085"},{"key":"e_1_3_3_1_43_2","doi-asserted-by":"crossref","unstructured":"Soumik Mondal and Patrick Bours. 2017. A study on continuous authentication using a combination of keystroke and mouse biometrics. Neurocomputing 230 (2017) 1\u201322.","DOI":"10.1016\/j.neucom.2016.11.031"},{"key":"e_1_3_3_1_44_2","unstructured":"Amazon MTurk. 2024. Amazon MTurk Participation Agreement. https:\/\/www.mturk.com\/worker\/participation-agreement\/. Online; Accessed: 2024-04-26."},{"key":"e_1_3_3_1_45_2","doi-asserted-by":"crossref","unstructured":"Kevin\u00a0J Mullinix Thomas\u00a0J Leeper James\u00a0N Druckman and Jeremy Freese. 2015. The generalizability of survey experiments. Journal of Experimental Political Science 2 2 (2015) 109\u2013138.","DOI":"10.1017\/XPS.2015.19"},{"key":"e_1_3_3_1_46_2","unstructured":"Government of Canada. 2024. Research Ethics Board: Overview. https:\/\/www.canada.ca\/en\/health-canada\/services\/science-research\/science-advice-decision-making\/research-ethics-board.html\/. Online; Accessed: 2024-04-01."},{"key":"e_1_3_3_1_47_2","unstructured":"openAI. 2024. openAI API. https:\/\/platform.openai.com\/docs\/api-reference\/introduction. Online; Accessed: 2024-07-24."},{"key":"e_1_3_3_1_48_2","doi-asserted-by":"crossref","unstructured":"Gabriele Paolacci and Jesse Chandler. 2014. Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science 23 3 (2014) 184\u2013188.","DOI":"10.1177\/0963721414531598"},{"key":"e_1_3_3_1_49_2","unstructured":"Puppeteer. 2024. Puppeteer. https:\/\/pptr.dev\/. Online; Accessed: 2024-07-24."},{"key":"e_1_3_3_1_50_2","unstructured":"PwnedPasswords. 2023. PwnedPasswordsDownloader. https:\/\/github.com\/HaveIBeenPwned\/PwnedPasswordsDownloader."},{"key":"e_1_3_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-69403-8_25"},{"key":"e_1_3_3_1_52_2","doi-asserted-by":"crossref","unstructured":"Christoph Schild Lau Lilleholt and Ingo Zettler. 2021. Behavior in cheating paradigms is linked to overall approval rates of crowdworkers. Journal of Behavioral Decision Making 34 2 (2021) 157\u2013166.","DOI":"10.1002\/bdm.2195"},{"key":"e_1_3_3_1_53_2","unstructured":"Selenium. 2024. WebDriver Selenium. https:\/\/www.selenium.dev\/documentation\/webdriver\/. Online; Accessed: 2024-07-24."},{"key":"e_1_3_3_1_54_2","doi-asserted-by":"crossref","unstructured":"Chao Shen Zhongmin Cai Xiaohong Guan Youtian Du and Roy\u00a0A Maxion. 2012. User authentication through mouse dynamics. IEEE Transactions on Information Forensics and Security 8 1 (2012) 16\u201330.","DOI":"10.1109\/TIFS.2012.2223677"},{"key":"e_1_3_3_1_55_2","unstructured":"SONA. 2024. Participant Pool Management for Universities SONA Systems. https:\/\/www.sona-systems.com\/. Online; Accessed: 2024-04-01."},{"key":"e_1_3_3_1_56_2","unstructured":"Chris Stokel-Walker. 2018. Bots on Amazon\u2019s Mechanical Turk are ruining psychology studies. https:\/\/www.newscientist.com\/article\/2176436-bots-on-amazons-mechanical-turk-are-ruining-psychology-studies\/. Online; Accessed: 2023-09-14."},{"key":"e_1_3_3_1_57_2","doi-asserted-by":"crossref","unstructured":"John Ternovski and Lilla Orr. 2022. A note on increases in inattentive online survey-takers since 2020. Journal of Quantitative Description: Digital Media 2 (2022) 1\u201335.","DOI":"10.51685\/jqd.2022.002"},{"key":"e_1_3_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/1501750.1501770"},{"key":"e_1_3_3_1_59_2","first-page":"463","volume-title":"24th USENIX Security Symposium (USENIX Security 15)","author":"Ur Blase","year":"2015","unstructured":"Blase Ur, Sean\u00a0M. Segreti, Lujo Bauer, Nicolas Christin, Lorrie\u00a0Faith Cranor, Saranga Komanduri, Darya Kurilova, Michelle\u00a0L. Mazurek, William Melicher, and Richard Shay. 2015. Measuring Real-World Accuracies and Biases in Modeling Password Guessability. In 24th USENIX Security Symposium (USENIX Security 15). USENIX Association, Washington, D.C., 463\u2013481."},{"key":"e_1_3_3_1_60_2","unstructured":"US Department of Justice Criminal Division. 2024. Criminal Division | Identity Theft. https:\/\/www.justice.gov\/criminal\/criminal-fraud\/identity-theft\/identity-theft-and-identity-fraud. Online; Accessed: 2024-04-26."},{"key":"e_1_3_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-99-7356-9_15"},{"key":"e_1_3_3_1_62_2","unstructured":"Margaret\u00a0A. Webb and June\u00a0P. Tangney. 2022. Too good to be true: Bots and bad data from Mechanical Turk. Perspectives on Psychological Science (2022) 17456916221120027."},{"key":"e_1_3_3_1_63_2","doi-asserted-by":"crossref","unstructured":"Dustin Wood Peter\u00a0D Harms Graham\u00a0H Lowman and Justin\u00a0A DeSimone. 2017. Response speed and response consistency as mutually validating indicators of data quality in online samples. Social Psychological and Personality Science 8 4 (2017) 454\u2013464.","DOI":"10.1177\/1948550617703168"},{"key":"e_1_3_3_1_64_2","doi-asserted-by":"crossref","unstructured":"Carol\u00a0M Woods. 2006. Careless responding to reverse-worded items: Implications for confirmatory factor analysis. Journal of Psychopathology and Behavioral Assessment 28 (2006) 186\u2013191.","DOI":"10.1007\/s10862-005-9004-7"}],"event":{"name":"NSPW '24: New Security Paradigms Workshop","location":"Bedford PA USA","acronym":"NSPW '24"},"container-title":["Proceedings of the New Security Paradigms Workshop"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3703465.3703472","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3703465.3703472","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:08Z","timestamp":1750295888000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3703465.3703472"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,16]]},"references-count":63,"alternative-id":["10.1145\/3703465.3703472","10.1145\/3703465"],"URL":"https:\/\/doi.org\/10.1145\/3703465.3703472","relation":{},"subject":[],"published":{"date-parts":[[2024,9,16]]},"assertion":[{"value":"2025-01-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}