{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T21:39:54Z","timestamp":1776116394549,"version":"3.50.1"},"reference-count":85,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW1","license":[{"start":{"date-parts":[[2024,4,17]],"date-time":"2024-04-17T00:00:00Z","timestamp":1713312000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2024,4,17]]},"abstract":"<jats:p>In this paper, we examine the work of data annotation. Specifically, we focus on the role of counting or quantification in organising annotation work. Based on an ethnographic study of data annotation in two outsourcing centres in India, we observe that counting practices and its associated logics are an integral part of day-to-day annotation activities. In particular, we call attention to the presumption of total countability observed in annotation - the notion that everything, from tasks, datasets and deliverables, to workers, work time, quality and performance, can be managed by applying the logics of counting. To examine this, we draw on sociological and socio-technical scholarship on quantification and develop the lens of a 'regime of counting' that makes explicit the specific counts, practices, actors and structures that underpin the pervasive counting in annotation. We find that within the AI supply chain and data work, counting regimes aid the assertion of authority by the AI clients (also called requesters) over annotation processes, constituting them as reductive, standardised, and homogenous. We illustrate how this has implications for i) how annotation work and workers get valued, ii) the role human discretion plays in annotation, and iii) broader efforts to introduce accountable and more just practices in AI. Through these implications, we illustrate the limits of operating within the logic of total countability. Instead, we argue for a view of counting as partial - located in distinct geographies, shaped by specific interests and accountable in only limited ways. This, we propose, sets the stage for a fundamentally different orientation to counting and what counts in data annotation.<\/jats:p>","DOI":"10.1145\/3637367","type":"journal-article","created":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T10:05:31Z","timestamp":1714385131000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":18,"title":["Making Data Work Count"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3299-3169","authenticated-orcid":false,"given":"Srravya","family":"Chandhiramowuli","sequence":"first","affiliation":[{"name":"University of Edinburgh, Edinburgh, United Kingdom"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6311-3967","authenticated-orcid":false,"given":"Alex S.","family":"Taylor","sequence":"additional","affiliation":[{"name":"University of Edinburgh, Edinburgh, United Kingdom"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6148-350X","authenticated-orcid":false,"given":"Sara","family":"Heitlinger","sequence":"additional","affiliation":[{"name":"City, University of London, London, United Kingdom"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5037-5017","authenticated-orcid":false,"given":"Ding","family":"Wang","sequence":"additional","affiliation":[{"name":"Google Research, Atlanta, GA, USA"}]}],"member":"320","published-online":{"date-parts":[[2024,4,26]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3555531"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/13835.003.0008"},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Sareeta Amrute Ranjit Singh and Rigoberto Lara Guzm\u00e1n. 2022. A Primer on AI in\/from the Majority World: An Empirical Site and a Standpoint. Technical Report. Data & Society Research Institute. https:\/\/dx.doi.org\/10.2139\/ssrn.4199467","DOI":"10.2139\/ssrn.4199467"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3432949"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3593013.3594114"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951715578951"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951716652159"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491101.3516502"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/6352.003.0013"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/6352.001.0001"},{"key":"e_1_2_1_11_1","volume-title":"Real and Rational Numbers: The American Imaginary of Countability. Social studies of science 31, 3","author":"Bowker Geoffrey C","year":"2001","unstructured":"Geoffrey C Bowker and Susan Leigh Star. 2001. Pure, Real and Rational Numbers: The American Imaginary of Countability. Social studies of science 31, 3 (2001), 422--425."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.24251\/HICSS.2023.251"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1086\/696137"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025510"},{"key":"e_1_2_1_15_1","volume-title":"Colonialism and Its Forms of Knowledge: The British in India","author":"Cohn Bernard","unstructured":"Bernard Cohn. 1996. Colonialism and Its Forms of Knowledge: The British in India. Princeton University Press, Princeton, NJ, USA."},{"key":"e_1_2_1_16_1","volume-title":"An Anthropologist among the Historians and Other Essays","author":"Cohn Bernard S","unstructured":"Bernard S Cohn. 1987. The Census, Social Structure and Objectification in South Asia. In An Anthropologist among the Historians and Other Essays. Oxford University Press, Oxford, UK, 224--254."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533213"},{"key":"e_1_2_1_18_1","volume-title":"The Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence","author":"Crawford Kate","unstructured":"Kate Crawford. 2021. The Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press, New Haven, CT, USA."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581376"},{"key":"e_1_2_1_20_1","volume-title":"Governance by Indicators: Global Power through Quantification and Rankings","author":"Davis Kevin","unstructured":"Kevin Davis, Angelina Fisher, Benedict Kingsbury, and Sally Engle Merry. 2012. Governance by Indicators: Global Power through Quantification and Rankings. Oxford University Press, Oxford, UK."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/7221.001.0001"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3593013.3593989"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1177\/103237329600100103"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1177\/1024258916687250"},{"key":"e_1_2_1_25_1","volume-title":"Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass","author":"Gray Mary L","year":"2019","unstructured":"Mary L Gray and Siddharth Suri. 2019. Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass. Harper-Collins Publishers, New York, NY, USA."},{"key":"e_1_2_1_26_1","volume-title":"Representing and Intervening: Introductory Topics in the Philosophy of Natural Science","author":"Hacking Ian","unstructured":"Ian Hacking. 1983. Representing and Intervening: Introductory Topics in the Philosophy of Natural Science. Cambridge University Press, Cambridge, UK."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511819766"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445918"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1177\/1461444813511926"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2470742"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274345"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1958824.1958861"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1177\/09526950022120665"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580645"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376166"},{"key":"e_1_2_1_36_1","volume-title":"Science in Action: How to Follow Scientists and Engineers through Society","author":"Latour Bruno","unstructured":"Bruno Latour. 1987. Science in Action: How to Follow Scientists and Engineers through Society. Harvard University Press, Cambridge, MA, USA."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1177\/20539517231188723"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3334480.3375158"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531663"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818052.2855514"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2750858.2807528"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1177\/103237320501000206"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-soc-073117-041343"},{"key":"e_1_2_1_44_1","volume-title":"Gender Violence, and Sex Trafficking. In The Seductions of Quantification","author":"Merry Sally Engle","unstructured":"Sally Engle Merry. 2016. The Seductions of Quantification: Measuring Human Rights, Gender Violence, and Sex Trafficking. In The Seductions of Quantification. University of Chicago Press, Chicago, IL, USA."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3555561"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3492853"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3415186"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1016\/0361-3682(87)90039-0"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1177\/1461444815604328"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-022-09434--7"},{"key":"e_1_2_1_51_1","volume-title":"The Mathematics of Death and Life After Genocide","author":"Nelson Diane M","unstructured":"Diane M Nelson. 2015. Who Counts?: The Mathematics of Death and Life After Genocide. Duke University Press, Durham, NC, USA."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1177\/103237329700200102"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1287\/orsc.13.6.684.501"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patter.2021.100336"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.14763\/2019.4.1425"},{"key":"e_1_2_1_56_1","volume-title":"Trust in Numbers","author":"Porter Theodore M","unstructured":"Theodore M Porter. 1996. Trust in Numbers. In Trust in Numbers. Princeton University Press, Princeton, NJ, USA."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1080\/1369118X.2022.2049849"},{"key":"e_1_2_1_58_1","unstructured":"Julian Posada. 2023. Platform Authority and Data Quality: Who Decides What Counts in Data Production for Artificial Intelligence? Technical Report. Berggruen Institute and Global Affairs Canada."},{"key":"e_1_2_1_59_1","volume-title":"The Coloniality of Data Work: Power and Inequality in Outsourced Data Production for Machine Learning. Ph. D. Dissertation","author":"Posada Gutierrez Julian Alberto","unstructured":"Julian Alberto Posada Gutierrez. 2022. The Coloniality of Data Work: Power and Inequality in Outsourced Data Production for Machine Learning. Ph. D. Dissertation. University of Toronto."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1177\/0018726704044955"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3449081"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1007\/978--1--84628--768--8"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-020-09388--8"},{"key":"e_1_2_1_64_1","volume-title":"Behind the Screen: Content Moderation in the Shadows of Social Media","author":"Roberts Sarah T","unstructured":"Sarah T Roberts. 2019. Behind the Screen: Content Moderation in the Shadows of Social Media. Yale University Press, New Haven, CT, USA."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/10993.003.0006"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580950"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1016\/0361-3682(91)90019-B"},{"key":"e_1_2_1_68_1","volume-title":"The World of Indicators: The Making of Governmental Knowledge through Quantification","author":"Rottenburg Richard","unstructured":"Richard Rottenburg, Sally E Merry, Sung-Joon Park, and Johanna Mugler. 2015. The World of Indicators: The Making of Governmental Knowledge through Quantification. Cambridge University Press, Cambridge, UK."},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/3555177"},{"key":"e_1_2_1_70_1","volume-title":"Census in Colonial India and the Birth of Caste. Economic and political weekly 46, 33","author":"Samarendra Padmanabh","year":"2011","unstructured":"Padmanabh Samarendra. 2011. Census in Colonial India and the Birth of Caste. Economic and political weekly 46, 33 (2011), 51--58."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3516515"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/3298735"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445518"},{"key":"e_1_2_1_74_1","volume-title":"The Politics of Numbers","author":"Starr Paul","unstructured":"Paul Starr. 1987. The Sociology of Official Statistics. In The Politics of Numbers. Russell Sage Foundation, New York, NY, USA, 7--57."},{"key":"e_1_2_1_75_1","volume-title":"Plans and Situated Actions: The Problem of Human-Machine Communication","author":"Suchman Lucy A","unstructured":"Lucy A Suchman. 1987. Plans and Situated Actions: The Problem of Human-Machine Communication. Cambridge University Press, Cambridge, UK."},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544549.3583178"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1025091532662"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/3476060"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951720919776"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0361541300003405"},{"key":"e_1_2_1_81_1","volume-title":"Science and an African Logic","author":"Verran Helen","unstructured":"Helen Verran. 2001. Science and an African Logic. University of Chicago Press, Chicago, IL, USA."},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376274"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3502121"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1177\/0950017018785616"},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501866"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3637367","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3637367","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T17:30:05Z","timestamp":1755883805000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3637367"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,17]]},"references-count":85,"journal-issue":{"issue":"CSCW1","published-print":{"date-parts":[[2024,4,17]]}},"alternative-id":["10.1145\/3637367"],"URL":"https:\/\/doi.org\/10.1145\/3637367","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,17]]},"assertion":[{"value":"2024-04-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}