{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:50:05Z","timestamp":1777456205231,"version":"3.51.4"},"reference-count":39,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2025,6,1]],"date-time":"2025-06-01T00:00:00Z","timestamp":1748736000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,6,1]],"date-time":"2025-06-01T00:00:00Z","timestamp":1748736000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/501100000286","name":"British Academy","doi-asserted-by":"publisher","award":["PF22\\220076"],"award-info":[{"award-number":["PF22\\220076"]}],"id":[{"id":"10.13039\/501100000286","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Big Data &amp; Society"],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:p>The production of artificial intelligence (AI) requires human labour, with tasks ranging from well-paid engineering work to often-outsourced data work. This commentary explores the economic and policy implications of improving working conditions for AI data workers, specifically focusing on the impact of clearer task instructions and increased pay for data annotators. It contrasts rule-based and standard-based approaches to task instructions, revealing evidence-based practices for increasing accuracy in annotation and lowering task difficulty for annotators. AI developers have an economic incentive to invest in these areas as better annotation can lead to higher quality AI systems. The findings have broader implications for AI policy beyond the fairness of labour standards in the AI economy. Testing the design of annotation instructions is crucial for the development of annotation standards as a prerequisite for scientific review and effective human oversight of AI systems in protection of ethical values and fundamental rights.<\/jats:p>","DOI":"10.1177\/20539517251351320","type":"journal-article","created":{"date-parts":[[2025,6,21]],"date-time":"2025-06-21T02:02:27Z","timestamp":1750471347000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":0,"title":["Better pay, clearer guidance: Investing in the working conditions of artificial intelligence data workers"],"prefix":"10.1177","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3043-075X","authenticated-orcid":false,"given":"Johann","family":"Laux","sequence":"first","affiliation":[{"name":"Oxford Internet Institute"},{"name":"Humboldt Institute for Internet and Society, Berlin, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0713-6010","authenticated-orcid":false,"given":"Fabian","family":"Stephany","sequence":"additional","affiliation":[{"name":"Oxford Internet Institute"},{"name":"Humboldt Institute for Internet and Society, Berlin, Germany"},{"name":"Bruegel, Brussels, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8580-6924","authenticated-orcid":false,"given":"Alice","family":"Liefgreen","sequence":"additional","affiliation":[{"name":"Department of Language and Cognition, University College London"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2025,6,20]]},"reference":[{"key":"e_1_3_2_2_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0274630"},{"issue":"4","key":"e_1_3_2_3_1","first-page":"1401","article-title":"The death of rules and standards","volume":"92","author":"Casey AJ","year":"2017","unstructured":"Casey AJ, Niblett A (2017) The death of rules and standards. Indiana Law Journal 92(4): 1401\u20131447.","journal-title":"Indiana Law Journal"},{"key":"e_1_3_2_4_1","doi-asserted-by":"publisher","DOI":"10.2196\/43251"},{"key":"e_1_3_2_5_1","doi-asserted-by":"publisher","DOI":"10.2307\/j.ctv1ghv45t"},{"key":"e_1_3_2_6_1","doi-asserted-by":"publisher","DOI":"10.1177\/20539517231203669"},{"key":"e_1_3_2_7_1","doi-asserted-by":"publisher","DOI":"10.1177\/20539517211035955"},{"key":"e_1_3_2_8_1","unstructured":"Eisenmann M Reinke A Weru V et al. (2023) Why is the Winner the Best?. https:\/\/doi.org\/10.48550\/ARXIV.2303.17719."},{"key":"e_1_3_2_9_1","unstructured":"Gebru T Morgenstern J Vecchione B et al. (2018) Datasheets for Datasets. https:\/\/doi.org\/10.48550\/ARXIV.1803.09010."},{"key":"e_1_3_2_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.techfore.2018.05.005"},{"key":"e_1_3_2_11_1","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.4603764"},{"key":"e_1_3_2_12_1","doi-asserted-by":"crossref","unstructured":"Graham M Anwar MA (2019) The global gig economy: Towards a planetary labour market? First Monday. https:\/\/doi.org\/10.5210\/fm.v24i4.9913","DOI":"10.5210\/fm.v24i4.9913"},{"key":"e_1_3_2_13_1","volume-title":"Ghost Work: How to Stop SiliconValley from Building a New Global Underclass","author":"Gray ML","year":"2019","unstructured":"Gray ML, Suri S (2019) Ghost Work: How to Stop SiliconValley from Building a New Global Underclass. Boston: Houghton Mifflin Harcourt."},{"key":"e_1_3_2_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.clsr.2022.105681"},{"key":"e_1_3_2_15_1","doi-asserted-by":"publisher","DOI":"10.2307\/1372840"},{"key":"e_1_3_2_16_1","first-page":"1661","volume-title":"Handbook of Public Economics","author":"Kaplow L","year":"2002","unstructured":"Kaplow L, Shavell S (2002) Economic analysis of law. In: Handbook of Public Economics. Amsterdam: Elsevier, 3, 1661\u20131784. https:\/\/doi.org\/10.1016\/S1573-4420(02)80029-5"},{"issue":"1","key":"e_1_3_2_17_1","first-page":"23","article-title":"Behavioral analysis and legal form: Rules vs. standards revisited","volume":"79","author":"Korobkin RB","year":"2000","unstructured":"Korobkin RB (2000) Behavioral analysis and legal form: Rules vs. standards revisited. Oregon Law Review 79(1): 23\u201360.","journal-title":"Oregon Law Review"},{"key":"e_1_3_2_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00146-023-01777-z"},{"key":"e_1_3_2_19_1","doi-asserted-by":"crossref","unstructured":"Laux J Ruschemeier H (2025) Automation Bias in the AI Act: On the Legal Implications of Attempting to De-Bias Human Oversight of AI. https:\/\/doi.org\/10.2139\/ssrn.5117560","DOI":"10.2139\/ssrn.5117560"},{"key":"e_1_3_2_20_1","unstructured":"Laux J Stephany F Liefgreen A (2023) Improving Task Instructions for Data Annotators: How Clear Rules and Higher Pay Increase Performance in Data Annotation in the AI Economy (Version 2). arXiv. https:\/\/doi.org\/10.48550\/ARXIV.2312.14565"},{"key":"e_1_3_2_21_1","doi-asserted-by":"publisher","DOI":"10.1177\/20539517231188723"},{"key":"e_1_3_2_22_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-014-0483-x"},{"key":"e_1_3_2_23_1","doi-asserted-by":"crossref","unstructured":"Miceli M Posada J (2022) The Data-Production Dispositif (Version 1). arXiv. https:\/\/doi.org\/10.48550\/ARXIV.2205.11963","DOI":"10.1145\/3555561"},{"issue":"1","key":"e_1_3_2_24_1","first-page":"205395172412326","article-title":"A typology of artificial intelligence data work","volume":"11","author":"Muldoon J","unstructured":"Muldoon J, Cant C, Wu B, et al. (2024a) A typology of artificial intelligence data work. Big Data & Society 11(1): 20539517241232632.","journal-title":"Big Data & Society"},{"key":"e_1_3_2_25_1","volume-title":"Feeding the Machine","author":"Muldoon J","unstructured":"Muldoon J, Graham M, Cant C (2024b) Feeding the Machine. Edinburgh: Canongate."},{"key":"e_1_3_2_26_1","unstructured":"Northcutt CG Athalye A Mueller J (2021) Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks. https:\/\/doi.org\/10.48550\/ARXIV.2103.14749"},{"key":"e_1_3_2_27_1","doi-asserted-by":"publisher","DOI":"10.15346\/hc.v7i1.1"},{"key":"e_1_3_2_28_1","unstructured":"Partnership on AI (n.d.) Improving Conditions for Data Enrichment Workers: Resources for AI Practitioners. Retrieved 3 October 2023 from https:\/\/partnershiponai.org\/responsible-sourcing-library\/"},{"key":"e_1_3_2_29_1","first-page":"107","volume-title":"Handbook of Statistics","author":"Patra GK","year":"2023","unstructured":"Patra GK, Bhimala KR, Marndi A, et al. (2023) Deep learning methods for scientific and industrial research. In: Handbook of Statistics. London: Elsevier, 48, 107\u2013168. https:\/\/doi.org\/10.1016\/bs.host.2022.12.002"},{"key":"e_1_3_2_30_1","doi-asserted-by":"crossref","unstructured":"Paullada A Raji ID Bender EM et al. (2020) Data and Its (Dis)contents: A Survey of Dataset Development and Use in Machine Learning Research. https:\/\/doi.org\/10.48550\/ARXIV.2012.05345","DOI":"10.1016\/j.patter.2021.100336"},{"key":"e_1_3_2_31_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-023-00625-5"},{"key":"e_1_3_2_32_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2022-071950"},{"key":"e_1_3_2_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.respol.2023.104898"},{"key":"e_1_3_2_34_1","doi-asserted-by":"crossref","unstructured":"Sterz S Baum K Biewer S et al. (2024) On the Quest for Effectiveness in Human Oversight: Interdisciplinary Perspectives. https:\/\/doi.org\/10.48550\/ARXIV.2404.04059","DOI":"10.1145\/3630106.3659051"},{"key":"e_1_3_2_35_1","doi-asserted-by":"publisher","DOI":"10.1177\/0038038520986082"},{"key":"e_1_3_2_36_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951720919776"},{"key":"e_1_3_2_37_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06221-2"},{"key":"e_1_3_2_38_1","unstructured":"Wong M (2023) America already has an AI underclass. The Atlantic. https:\/\/www.theatlantic.com\/technology\/archive\/2023\/07\/ai-chatbot-human-evaluator-feedback\/674805\/"},{"key":"e_1_3_2_39_1","doi-asserted-by":"publisher","DOI":"10.1177\/0950017018785616"},{"key":"e_1_3_2_40_1","doi-asserted-by":"crossref","unstructured":"Yang K Qinami K Fei-Fei L et al. (2020) Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the ImageNet hierarchy. In Proceedings of the 2020 Conference on Fairness Accountability and Transparency pp. 547\u2013558. https:\/\/doi.org\/10.1145\/3351095.3375709","DOI":"10.1145\/3351095.3375709"}],"container-title":["Big Data &amp; Society"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/20539517251351320","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/20539517251351320","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/20539517251351320","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T13:01:16Z","timestamp":1777381276000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/20539517251351320"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6]]},"references-count":39,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["10.1177\/20539517251351320"],"URL":"https:\/\/doi.org\/10.1177\/20539517251351320","relation":{},"ISSN":["2053-9517","2053-9517"],"issn-type":[{"value":"2053-9517","type":"print"},{"value":"2053-9517","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6]]},"article-number":"20539517251351320"}}