{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T02:10:28Z","timestamp":1760667028026,"version":"build-2065373602"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"7","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2025,10,18]]},"abstract":"<jats:p>ChatGPT and other large language models (LLMs) have proven useful in crowdsourcing tasks, where they can effectively annotate machine learning training data. However, this means that they also have the potential for misuse, specifically to automatically answer surveys. LLMs can potentially circumvent quality assurance measures, thereby threatening the integrity of methodologies that rely on crowdsourcing surveys. In this paper, we propose a mechanism to detect LLM-generated responses to surveys. The mechanism uses ''prompt injection,'' such as directions that can mislead LLMs into giving predictable responses. We evaluate our technique against a range of question scenarios, types, and positions, and find that it can reliably detect LLM-generated responses with more than 98% effectiveness. We also provide an open-source software to help survey designers use our technique to detect LLM responses. Our work is a step in ensuring that survey methodologies remain rigorous vis-a-vis LLMs.<\/jats:p>","DOI":"10.1145\/3757503","type":"journal-article","created":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T16:59:10Z","timestamp":1760633950000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Safeguarding Crowdsourcing Surveys from ChatGPT through Prompt Injection"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8213-6582","authenticated-orcid":false,"given":"Chaofan","family":"Wang","sequence":"first","affiliation":[{"name":"Wenzhou University, Wenzhou, China and Delft University of Technology, Delft, Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8684-0585","authenticated-orcid":false,"given":"Samuel","family":"Kernan Freire","sequence":"additional","affiliation":[{"name":"Hague University of Applied Sciences, The Hague, Netherlands and Delft University of Technology, Delft, Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7302-9088","authenticated-orcid":false,"given":"Mo","family":"Zhang","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Melbourne, VIC, Australia and University of Birmingham, Birmingham, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8522-8607","authenticated-orcid":false,"given":"Jing","family":"Wei","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Melbourne, VIC, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0117-0322","authenticated-orcid":false,"given":"Jorge","family":"Goncalves","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Melbourne, VIC, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2804-6038","authenticated-orcid":false,"given":"Vassilis","family":"Kostakos","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Melbourne, VIC, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3300-2913","authenticated-orcid":false,"given":"Alessandro","family":"Bozzon","sequence":"additional","affiliation":[{"name":"Delft University of Technology, Delft, Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0484-4214","authenticated-orcid":false,"given":"Evangelos","family":"Niforatos","sequence":"additional","affiliation":[{"name":"Delft University of Technology, Delft, Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,10,16]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Zeljana Basic Ana Banovac Ivana Kruzic and Ivan Jerkovic. 2023. Better by you better than me chatgpt3 as writing assistance in students essays. arXiv:2302.04536 [cs.AI]","DOI":"10.35542\/osf.io\/n5m7s"},{"volume-title":"Natural language processing with Python: analyzing text with the natural language toolkit. ''O'Reilly Media","author":"Bird Steven","key":"e_1_2_1_2_1","unstructured":"Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python: analyzing text with the natural language toolkit. ''O'Reilly Media, Inc.''."},{"key":"e_1_2_1_3_1","volume-title":"Bad Characters: Imperceptible NLP Attacks. In 43rd IEEE Symposium on Security and Privacy. IEEE.","author":"Boucher Nicholas","year":"2022","unstructured":"Nicholas Boucher, Ilia Shumailov, Ross Anderson, and Nicolas Papernot. 2022. Bad Characters: Imperceptible NLP Attacks. In 43rd IEEE Symposium on Security and Privacy. IEEE."},{"key":"e_1_2_1_4_1","unstructured":"Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. Advances in neural information processing systems Vol. 33 (2020) 1877-1901."},{"key":"e_1_2_1_5_1","doi-asserted-by":"crossref","unstructured":"Jan Cegin Jakub Simko and Peter Brusilovsky. 2023. ChatGPT to Replace Crowdsourcing of Paraphrases for Intent Classification: Higher Diversity and Comparable Model Robustness. arXiv:2305.12947 [cs.CL]","DOI":"10.18653\/v1\/2023.emnlp-main.117"},{"key":"e_1_2_1_6_1","unstructured":"Paul Christiano Jan Leike Tom B. Brown Miljan Martic Shane Legg and Dario Amodei. 2023. Deep reinforcement learning from human preferences. arXiv:1706.03741 [stat.ML]"},{"key":"e_1_2_1_7_1","volume-title":"University of California","author":"Committee for Protection of Human Subjects","year":"2023","unstructured":"Committee for Protection of Human Subjects, University of California, Berkeley. 2023. DECEPTION AND INCOMPLETE DISCLOSURE IN RESEARCH. https:\/\/cphs.berkeley.edu\/deception.pdf\/ Accessed: 2025-02-16."},{"key":"e_1_2_1_8_1","doi-asserted-by":"crossref","unstructured":"Evan Crothers Nathalie Japkowicz and Herna Viktor. 2023. Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods. arXiv:2210.07321 [cs.CL]","DOI":"10.1109\/ACCESS.2023.3294090"},{"key":"e_1_2_1_9_1","volume-title":"H Chi, and Steven Dow","author":"Egelman Serge","year":"2014","unstructured":"Serge Egelman, Ed H Chi, and Steven Dow. 2014. Crowdsourcing in HCI research. Ways of Knowing in HCI (2014), 267-289."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11023-020-09548-1"},{"key":"e_1_2_1_11_1","unstructured":"George Denison. 2023. LLM use in research: A study into mitigation strategies. https:\/\/www.prolific.com\/resources\/llm-use-in-research-a-study-into-mitigation-strategies"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1111\/ijmr.12135"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1093\/jcr\/ucx047"},{"key":"e_1_2_1_14_1","unstructured":"Riley Goodside. 2022. Exploiting GPT-3 prompts with malicious inputs that order the model to ignore its previous directions. https:\/\/twitter.com\/goodside\/status\/1569128808308957185."},{"key":"e_1_2_1_15_1","unstructured":"Kai Greshake Sahar Abdelnabi Shailesh Mishra Christoph Endres Thorsten Holz and Mario Fritz. 2023. More than you've asked for: A Comprehensive Analysis of Novel Prompt Injection Threats to Application-Integrated Large Language Models. arXiv:2302.12173 [cs.CR]"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580688"},{"key":"e_1_2_1_17_1","volume-title":"Nan Duan, and Weizhu Chen.","author":"He Xingwei","year":"2023","unstructured":"Xingwei He, Zhenghao Lin, Yeyun Gong, A-Long Jin, Hang Zhang, Chen Lin, Jian Jiao, Siu Ming Yiu, Nan Duan, and Weizhu Chen. 2023. AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators. arXiv:2303.16854 [cs.CL]"},{"key":"e_1_2_1_18_1","unstructured":"Krystal Hu. 2023. ChatGPT sets record for fastest-growing user base - analyst note. https:\/\/www.reuters.com\/technology\/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01\/."},{"key":"e_1_2_1_19_1","volume-title":"Institutional Review Board","author":"Protection Program Human Research","year":"2025","unstructured":"Human Research Protection Program, Institutional Review Board. 2025. Research Involving Deception. https:\/\/research.oregonstate.edu\/ori\/irb\/research-involving-deception\/ Accessed: 2025-02-16."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.nlp.2023.100020"},{"key":"e_1_2_1_21_1","volume-title":"Johanna Topalis, Tobias Weber, Philipp Wesp, Bastian Sabel, Jens Ricke, and Michael Ingrisch.","author":"Jeblick Katharina","year":"2022","unstructured":"Katharina Jeblick, Balthasar Schachtner, Jakob Dexl, Andreas Mittermeier, Anna Theresa St\u00fcber, Johanna Topalis, Tobias Weber, Philipp Wesp, Bastian Sabel, Jens Ricke, and Michael Ingrisch. 2022. ChatGPT Makes Medicine Easy to Swallow: An Exploratory Case Study on Simplified Radiology Reports. arXiv:2212.14882 [cs.CL]"},{"key":"e_1_2_1_22_1","volume-title":"Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa.","author":"Kojima Takeshi","year":"2023","unstructured":"Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2023. Large Language Models are Zero-Shot Reasoners. arXiv:2205.11916 [cs.CL]"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1111\/apps.12108"},{"key":"e_1_2_1_24_1","volume-title":"33rd USENIX Security Symposium (USENIX Security 24)","author":"Liu Yupei","year":"2024","unstructured":"Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. 2024. Formalizing and benchmarking prompt injection attacks and defenses. In 33rd USENIX Security Symposium (USENIX Security 24). 1831-1847."},{"key":"e_1_2_1_25_1","doi-asserted-by":"crossref","unstructured":"Kamil Malinka Martin Peres\u00edni Anton Firc and Filip Janus. 2023. On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree? arXiv:2303.11146 [cs.CY]","DOI":"10.1145\/3587102.3588827"},{"key":"e_1_2_1_26_1","volume-title":"Do prompt positions really matter? arXiv preprint arXiv:2305.14493","author":"Mao Junyu","year":"2023","unstructured":"Junyu Mao, Stuart E Middleton, and Mahesan Niranjan. 2023. Do prompt positions really matter? arXiv preprint arXiv:2305.14493 (2023)."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1002\/sec.1157"},{"key":"e_1_2_1_28_1","doi-asserted-by":"crossref","unstructured":"Oded Nov Nina Singh and Devin Mann. 2023. Putting ChatGPT's Medical Advice to the (Turing) Test. arXiv:2301.10035 [cs.HC]","DOI":"10.1101\/2023.01.23.23284735"},{"key":"e_1_2_1_29_1","volume-title":"University of Connecticut","author":"Vice Office","year":"2009","unstructured":"Office of the Vice President for Research, University of Connecticut. 2009. Researcher's Guide: Deception\/Debriefing. https:\/\/ovpr.uconn.edu\/services\/rics\/irb\/researcher-guide\/deception\/"},{"key":"e_1_2_1_30_1","unstructured":"OpenAI. 2023. ChatGPT: Optimizing Language Models for Dialogue. https:\/\/openai.com\/blog\/chatgpt\/."},{"key":"e_1_2_1_31_1","unstructured":"OpenAI. 2024. GPT-4o. https:\/\/openai.com\/index\/hello-gpt-4o\/."},{"key":"e_1_2_1_32_1","volume-title":"Using deception ethically: Practical research guidelines for researchers and reviewers. Canadian Psychology\/psychologie canadienne","author":"Pascual-Leone Antonio","year":"2010","unstructured":"Antonio Pascual-Leone, Terence Singh, and Alan Scoboria. 2010. Using deception ethically: Practical research guidelines for researchers and reviewers. Canadian Psychology\/psychologie canadienne, Vol. 51, 4 (2010), 241."},{"key":"e_1_2_1_33_1","unstructured":"F\u00e1bio Perez and Ian Ribeiro. 2022. Ignore Previous Prompt: Attack Techniques For Language Models. arXiv:2211.09527 [cs.CL]"},{"key":"e_1_2_1_34_1","unstructured":"Prolific. 2024. How to detect and prevent the use of Large Language Models in studies. https:\/\/researcher-help.prolific.com\/en\/article\/2a85ea"},{"key":"e_1_2_1_35_1","unstructured":"Qwen Team. 2024. Qwen2.5: A Party of Foundation Models. https:\/\/qwenlm.github.io\/blog\/qwen2.5\/"},{"key":"e_1_2_1_36_1","volume-title":"Prompt Engineering Guide. https:\/\/github.com\/dair-ai\/Prompt-Engineering-Guide (12","author":"Saravia Elvis","year":"2022","unstructured":"Elvis Saravia. 2022. Prompt Engineering Guide. https:\/\/github.com\/dair-ai\/Prompt-Engineering-Guide (12 2022)."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1093\/cid\/cit005"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3637371"},{"key":"e_1_2_1_39_1","unstructured":"Ruixiang Tang Xiaotian Han Xiaoqian Jiang and Xia Hu. 2023. Does Synthetic Data Generation of LLMs Help Clinical Text Mining? arXiv:2303.04360 [cs.CL]"},{"key":"e_1_2_1_40_1","unstructured":"Dietrich Trautmann Alina Petrova and Frank Schilder. 2022. Legal Prompt Engineering for Multilingual Legal Judgement Prediction. arXiv:2212.02199 [cs.CL]"},{"key":"e_1_2_1_41_1","unstructured":"Jason Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph Sebastian Borgeaud Dani Yogatama Maarten Bosma Denny Zhou Donald Metzler et al. 2022. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 (2022)."},{"key":"e_1_2_1_42_1","volume-title":"Chi, Quoc Le, and Denny Zhou","author":"Wei Jason","year":"2023","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903 [cs.CL]"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAU.1967.1161901"},{"key":"e_1_2_1_44_1","volume-title":"A survey on llm-gernerated text detection: Necessity, methods, and future directions. arXiv preprint arXiv:2310.14724","author":"Wu Junchao","year":"2023","unstructured":"Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Derek F Wong, and Lidia S Chao. 2023. A survey on llm-gernerated text detection: Necessity, methods, and future directions. arXiv preprint arXiv:2310.14724 (2023)."},{"key":"e_1_2_1_45_1","doi-asserted-by":"crossref","unstructured":"Mengjie Zhao Fei Mi Yasheng Wang Minglei Li Xin Jiang Qun Liu and Hinrich Sch\u00fctze. 2022. LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework. arXiv:2112.07522 [cs.CL]","DOI":"10.18653\/v1\/2022.findings-naacl.51"},{"key":"e_1_2_1_46_1","unstructured":"Wayne Xin Zhao Kun Zhou Junyi Li Tianyi Tang Xiaolei Wang Yupeng Hou Yingqian Min Beichen Zhang Junjie Zhang Zican Dong Yifan Du Chen Yang Yushuo Chen Zhipeng Chen Jinhao Jiang Ruiyang Ren Yifan Li Xinyu Tang Zikang Liu Peiyu Liu Jian-Yun Nie and Ji-Rong Wen. 2023. A Survey of Large Language Models. arXiv:2303.18223 [cs.CL]"},{"key":"e_1_2_1_47_1","unstructured":"Mingkai Zheng Xiu Su Shan You Fei Wang Chen Qian Chang Xu and Samuel Albanie. 2023. Can GPT-4 Perform Neural Architecture Search? arXiv:2304.10970 [cs.LG]"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3757503","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T01:54:48Z","timestamp":1760666088000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3757503"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,16]]},"references-count":47,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,10,18]]}},"alternative-id":["10.1145\/3757503"],"URL":"https:\/\/doi.org\/10.1145\/3757503","relation":{},"ISSN":["2573-0142"],"issn-type":[{"type":"electronic","value":"2573-0142"}],"subject":[],"published":{"date-parts":[[2025,10,16]]},"assertion":[{"value":"2025-10-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}