{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T18:47:42Z","timestamp":1776106062680,"version":"3.50.1"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","issue":"7","funder":[{"name":"University of Rome La Sapienza","award":["RG123188B3EF6A80"],"award-info":[{"award-number":["RG123188B3EF6A80"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2025,10,18]]},"abstract":"<jats:p>\n            In microtask crowdsourcing, Human Intelligence Tasks (HITs) are commonly allocated on a first-come, first-served basis: they are published on the platform and the fastest workers select the most attractive ones first. This step has not received much attention from the scientific community yet, though it can become particularly taxing for workers when they compete to secure the most sought-after tasks. There are many strategies to ensure one's access to tasks and their effects on the labour process as a whole are not well understood. For instance, platforms with a sizeable task reservation queue allow workers to gain preferential access to a large number of tasks, which in turn may cause a shortage of work for the rest of the crowd. For the requesters, this means lower rates of completion and a lack of worker diversity. We explore workers' strategies for accessing and reserving tasks using monitoring techniques from both client and server sides. We investigate how these strategies affect task execution, in terms of availability, completion time, and answer quality, by deploying 1000 image annotation HITs in Amazon Mechanical Turk including objective and subjective tasks. We observe that workers who do not use automated catching techniques tend to have higher annotation quality, are more focused, spend more effort on text editing, and provide a higher diversity of output than workers using such tools. This study also reveals the\n            <jats:italic toggle=\"yes\">tragedy of the commons<\/jats:italic>\n            effect among platform members due to the use of catching techniques: workers using automated catching techniques reserve and complete a substantially higher portion of the available tasks, but the over-reservation of HITs restricts all workers of reservation opportunities, and compromise their own future labour capacity as well. We observe a high inefficiency in job completions, as the majority of the times a task is being reserved by a worker, it will not get actually performed and will need to be republished for further allocation. Finally, we propose solutions to mitigate the negative effects of these phenomena on the labour process.\n          <\/jats:p>","DOI":"10.1145\/3757426","type":"journal-article","created":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T17:06:01Z","timestamp":1760634361000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Sorry, Your HIT Is Overbooked - Investigating the Use of Crowdsourcing HIT Catchers"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5423-8669","authenticated-orcid":false,"given":"Eddy","family":"Maddalena","sequence":"first","affiliation":[{"name":"University of Udine, Udine, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0981-3409","authenticated-orcid":false,"given":"Alessandro","family":"Checco","sequence":"additional","affiliation":[{"name":"University of Rome La Sapienza, Rome, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9967-0626","authenticated-orcid":false,"given":"Haoyu","family":"Xie","sequence":"additional","affiliation":[{"name":"University of Sheffield, Sheffield, United Kingdom"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3110-7495","authenticated-orcid":false,"given":"Efpraxia","family":"Zamani","sequence":"additional","affiliation":[{"name":"Durham University, Durham, United Kingdom"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1722-947X","authenticated-orcid":false,"given":"Elena","family":"Simperl","sequence":"additional","affiliation":[{"name":"King's College London, London, United Kingdom"}]}],"member":"320","published-online":{"date-parts":[[2025,10,16]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"The micro-task market for lemons: Data quality on Amazon's Mechanical Turk. Political Science Research and Methods","author":"Ahler Douglas J","unstructured":"Douglas J Ahler, Carolyn E Roush, and Gaurav Sood. 2019. The micro-task market for lemons: Data quality on Amazon's Mechanical Turk. Political Science Research and Methods, Vol. First View (2019), 1-20."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1177\/0309816819899410"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1177\/1024529420914473"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-96180-0_18"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1108\/JICES-08-2022-0069"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.352.6291.1263"},{"key":"e_1_2_1_7_1","volume-title":"Online instrument delivery and participant recruitment services: Emerging opportunities for behavioral accounting research. Behavioral Research in Accounting","author":"Brandon Duane M","year":"2014","unstructured":"Duane M Brandon, James H Long, Tina M Loraas, Jennifer Mueller-Phillips, and Brian Vansant. 2014. Online instrument delivery and participant recruitment services: Emerging opportunities for behavioral accounting research. Behavioral Research in Accounting, Vol. 26, 1 (2014), 1-23. Publisher: American Accounting Assocation."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.11332"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1177\/1948550619875149"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.hrmr.2020.100762"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.350"},{"key":"e_1_2_1_12_1","volume-title":"Principles of applied statistics","author":"Cox David Roxbee","unstructured":"David Roxbee Cox and Christl A Donnelly. 2011. Principles of applied statistics. Cambridge University Press."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675260"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3148148"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1057\/s41265-016-0033-3"},{"key":"e_1_2_1_16_1","volume-title":"Is crowdsourcing a source of worker empowerment or exploitation? understanding crowd workers' perceptions of crowdsourcing career. Citeseer","author":"Deng Xuefei Nancy","year":"2013","unstructured":"Xuefei Nancy Deng and KD Joshi. 2013. Is crowdsourcing a source of worker empowerment or exploitation? understanding crowd workers' perceptions of crowdsourcing career. Citeseer (2013)."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v2i1.13154"},{"key":"e_1_2_1_18_1","first-page":"26","article-title":"Mechanical cheat: Spamming schemes and adversarial techniques on crowdsourcing platforms. In CrowdSearch. Citeseer, CEUR-WS.org, Lyon","author":"Difallah Djellel Eddine","year":"2012","unstructured":"Djellel Eddine Difallah, Gianluca Demartini, and Philippe Cudr\u00e9-Mauroux. 2012. Mechanical cheat: Spamming schemes and adversarial techniques on crowdsourcing platforms. In CrowdSearch. Citeseer, CEUR-WS.org, Lyon, France, 26-30.","journal-title":"France"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.4169\/000298910x523344"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1257\/jep.33.4.211"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3078714.3078715"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2305016120"},{"key":"e_1_2_1_23_1","volume-title":"Internet research in psychology. Annual review of psychology","author":"Gosling Samuel D","year":"2015","unstructured":"Samuel D Gosling and Winter Mason. 2015. Internet research in psychology. Annual review of psychology, Vol. 66 (2015), 877-902. Publisher: Annual Reviews."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v6i1.13331"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3291035"},{"key":"e_1_2_1_26_1","article-title":"The impact of task abandonment in crowdsourcing","volume":"33","author":"Han Lei","year":"2019","unstructured":"Lei Han, Kevin Roitero, Ujwal Gadiraju, Cristina Sarasua, Alessandro Checco, Eddy Maddalena, and Gianluca Demartini. 2019b. The impact of task abandonment in crowdsourcing. IEEE Transactions on Knowledge and Data Engineering, Vol. 33 (2019).","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-018-9312-6"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.162.3859.1243"},{"key":"e_1_2_1_29_1","volume-title":"1968 b. The tragedy of the commons: the population problem has no technical solution","author":"Hardin Garrett","year":"1968","unstructured":"Garrett Hardin. 1968 b. The tragedy of the commons: the population problem has no technical solution; it requires a fundamental extension in morality. science, Vol. 162, 3859 (1968), 1243-1248."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753357"},{"key":"e_1_2_1_31_1","unstructured":"Brandon Hellman. 2023. MTurk Suite. https:\/\/github.com\/Kadauchi\/mturk-suite"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1080\/00140138908966848"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/CCE.2014.6916756"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2470742"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300621"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v6i1.13327"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1017\/psrm.2020.6"},{"key":"e_1_2_1_38_1","volume-title":"Cognitive structure, flexibility, and plasticity in human multitasking-An integrative review of dual-task and task-switching research. Psychological bulletin","author":"Koch Iring","year":"2018","unstructured":"Iring Koch, Edita Poljac, Hermann M\u00fcller, and Andrea Kiesel. 2018. Cognitive structure, flexibility, and plasticity in human multitasking-An integrative review of dual-task and task-switching research. Psychological bulletin, Vol. 144, 6 (2018), 557. Publisher: American Psychological Association."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.650"},{"key":"e_1_2_1_40_1","volume-title":"An Interview with Elinor Ostrom. Annual Reviews Conversations","author":"Levi Margaret","year":"2010","unstructured":"Margaret Levi. 2010. An Interview with Elinor Ostrom. Annual Reviews Conversations (2010), 1-13."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-demo.1"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v4i1.13284"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3604940"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3403931"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531663"},{"key":"e_1_2_1_46_1","volume-title":"Conducting behavioral research on Amazon's Mechanical Turk. Behavior research methods","author":"Mason Winter","year":"2012","unstructured":"Winter Mason and Siddharth Suri. 2012. Conducting behavioral research on Amazon's Mechanical Turk. Behavior research methods, Vol. 44, 1 (2012), 1-23."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1108\/EJM-07-2017-0491"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858539"},{"key":"e_1_2_1_49_1","volume-title":"Patterns","volume":"1","author":"Midway Stephen R","year":"2020","unstructured":"Stephen R Midway. 2020. Principles of effective data visualization. Patterns, Vol. 1, 9 (2020)."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2016.2619901"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1038\/nmeth.1938"},{"key":"e_1_2_1_52_1","volume-title":"Crowdwork and the mobile underclass: Barriers to participation in India and the United States. new media & society","author":"Newlands Gemma","year":"2021","unstructured":"Gemma Newlands and Christoph Lutz. 2021. Crowdwork and the mobile underclass: Barriers to participation in India and the United States. new media & society, Vol. 23, 6 (2021), 1341-1361."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbef.2017.12.004"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1177\/1460458220907435"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.1.5697.692-c"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2011.5946971"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047199"},{"key":"e_1_2_1_58_1","first-page":"1241","volume-title":"Proceedings of The Web Conference","author":"Savage Saiph","year":"2020","unstructured":"Saiph Savage, Chun Wei Chiang, Susumu Saito, Carlos Toxtli, and Jeffrey Bigham. 2020. Becoming the super turker: Increasing wages via a strategy from high earning workers. In Proceedings of The Web Conference 2020. Association for Computing Machinery, New York, NY, USA, 1241-1252."},{"key":"e_1_2_1_59_1","volume-title":"LDA and WAM. In 2024 International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS). IEEE, 01-07","author":"Subramanian D Venkata","year":"2024","unstructured":"D Venkata Subramanian, J Thangakumar, M Preetha, S Ganga, and Suraj Sajeev. 2024. Similarities and Ranking of Documents Using TF-IDF, LDA and WAM. In 2024 International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS). IEEE, 01-07."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1111\/ajsp.12477"},{"key":"e_1_2_1_61_1","unstructured":"Alexandru C Telea. 2014. Data visualization: principles and practice. cRC Press."},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3476060"},{"key":"e_1_2_1_63_1","volume-title":"Manoel Horta Ribeiro, and Robert West","author":"Veselovsky Veniamin","year":"2023","unstructured":"Veniamin Veselovsky, Manoel Horta Ribeiro, and Robert West. 2023. Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks. arXiv:2306.07899 [cs.CL]"},{"key":"e_1_2_1_64_1","volume-title":"Are Mechanical Turk worker samples representative of health status and health behaviors in the US? PloS one","author":"Walters Kelly","year":"2018","unstructured":"Kelly Walters, Dimitri A Christakis, and Davene R Wright. 2018. Are Mechanical Turk worker samples representative of health status and health behaviors in the US? PloS one, Vol. 13, 6 (2018), e0198835. Publisher: Public Library of Science San Francisco, CA USA."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3027063.3053150"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359126"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.656"},{"key":"e_1_2_1_68_1","volume-title":"The Unintended Consequences of Automated Scripts in Crowdwork Platforms: A Simulation Study in MTurk. Information Systems Frontiers","author":"Xie Haoyu","year":"2023","unstructured":"Haoyu Xie, Alessandro Checco, and Efpraxia D Zamani. 2023a. The Unintended Consequences of Automated Scripts in Crowdwork Platforms: A Simulation Study in MTurk. Information Systems Frontiers (2023), 1-17."},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-023-09464-9"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3757426","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T01:55:29Z","timestamp":1760666129000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3757426"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,16]]},"references-count":69,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,10,18]]}},"alternative-id":["10.1145\/3757426"],"URL":"https:\/\/doi.org\/10.1145\/3757426","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,16]]},"assertion":[{"value":"2025-10-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}