{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T17:34:43Z","timestamp":1782408883334,"version":"3.54.5"},"reference-count":81,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2020,10,14]],"date-time":"2020-10-14T00:00:00Z","timestamp":1602633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"crossref","award":["16DII113"],"award-info":[{"award-number":["16DII113"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2020,10,14]]},"abstract":"<jats:p>The interpretation of data is fundamental to machine learning. This paper investigates practices of image data annotation as performed in industrial contexts. We define data annotation as a sense-making practice, where annotators assign meaning to data through the use of labels. Previous human-centered investigations have largely focused on annotators? subjectivity as a major cause of biased labels. We propose a wider view on this issue: guided by constructivist grounded theory, we conducted several weeks of fieldwork at two annotation companies. We analyzed which structures, power relations, and naturalized impositions shape the interpretation of data. Our results show that the work of annotators is profoundly informed by the interests, values, and priorities of other actors above their station. Arbitrary classifications are vertically imposed on annotators, and through them, on data. This imposition is largely naturalized. Assigning meaning to data is often presented as a technical matter. This paper shows it is, in fact, an exercise of power with multiple implications for individuals and society.<\/jats:p>","DOI":"10.1145\/3415186","type":"journal-article","created":{"date-parts":[[2020,10,15]],"date-time":"2020-10-15T18:27:49Z","timestamp":1602786469000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":136,"title":["Between Subjectivity and Imposition"],"prefix":"10.1145","volume":"4","author":[{"given":"Milagros","family":"Miceli","sequence":"first","affiliation":[{"name":"Technische Universit\u00e4t Berlin \/ Weizenbaum Institute, Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Schuessler","sequence":"additional","affiliation":[{"name":"Technische Universit\u00e4t Berlin \/ Weizenbaum Institute, Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tianling","family":"Yang","sequence":"additional","affiliation":[{"name":"Technische Universit\u00e4t Berlin \/ Weizenbaum Institute, Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,10,15]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359301"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300760"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.indmarman.2006.05.013"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1080\/17405904.2012.744320"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372859"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.15779\/Z38BG31"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00041"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.2752\/175630615X14212498964196"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511812507"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00174048"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.2307\/202060"},{"key":"e_1_2_1_12_1","volume-title":"The logic of practice reprinted ed.)","author":"Bourdieu Pierre","unstructured":"Pierre Bourdieu. 1990. The logic of practice reprinted ed.). Polity Press, Cambridge."},{"key":"e_1_2_1_13_1","volume-title":"Language and Symbolic Power new ed.)","author":"Bourdieu Pierre","unstructured":"Pierre Bourdieu. 1992. Language and Symbolic Power new ed.). Blackwell Publishers, Cambridge."},{"key":"e_1_2_1_14_1","unstructured":"Pierre Bourdieu. 2000. Pascalian Meditations .Stanford University Press Stanford Calif."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1177\/030631200030005001"},{"key":"e_1_2_1_16_1","volume-title":"Bowker and Susan Leigh Star","author":"Geoffrey","year":"1999","unstructured":"Geoffrey C. Bowker and Susan Leigh Star. 1999. Sorting Things out: Classification and Its Consequences .MIT Press, Cambridge, Mass. BD175 .B68 1999"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1080\/1369118X.2012.678878"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.606"},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the 1st Conference on Fairness, Accountability and Transparency","volume":"81","author":"Buolamwini Joy","year":"2018","unstructured":"Joy Buolamwini and Timnit Gebru. 2018. Gender Shades : Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency, Vol. 81. PMLR, 77--91."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1111\/anti.12534"},{"key":"e_1_2_1_21_1","volume-title":"Constructing Grounded Theory : A Practical Guide through Qualitative Analysis","author":"Charmaz Kathy","year":"2006","unstructured":"Kathy Charmaz. 2006. Constructing Grounded Theory : A Practical Guide through Qualitative Analysis .Sage Publications, London ; Thousand Oaks, Calif. H61.24 .C45 2006"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2481492.2481519"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2908216.2908220"},{"key":"e_1_2_1_24_1","article-title":"The Scored Society : Due Process for Automated Predictions","volume":"89","author":"Citron Danielle Keats","year":"2014","unstructured":"Danielle Keats Citron and Frank Pasquale. 2014. The Scored Society : Due Process for Automated Predictions. Washington Law Review, Vol. 89, 1 (2014).","journal-title":"Washington Law Review"},{"key":"e_1_2_1_25_1","volume-title":"Strauss","author":"Corbin Juliet M.","year":"2015","unstructured":"Juliet M. Corbin and Anselm L. Strauss. 2015. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory fourth edition ed.). SAGE, Los Angeles. HA29 .C7724 2015"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1177\/1527476418796632"},{"key":"e_1_2_1_27_1","unstructured":"Kate Crawford and Trevor Paglen. 2019. Excavating AI. https:\/\/www.excavating.ai."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1177\/019145379602200603"},{"key":"e_1_2_1_29_1","unstructured":"Hannah Davis. 2020. A Dataset Is a Worldview. https:\/\/towardsdatascience.com\/a-dataset-is-a-worldview-5328216dd44d."},{"key":"e_1_2_1_30_1","volume-title":"Klein","author":"D'Ignazio Catherine","year":"2020","unstructured":"Catherine D'Ignazio and Lauren F. Klein. 2020. Data Feminism .The MIT Press, Cambridge, Massachusetts. HQ1190 .D574 2020"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3373157"},{"key":"e_1_2_1_32_1","unstructured":"Emile Durkheim and Marcel Mauss. 1963. Primitive Classification. University of Chicago Press."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1080\/03637751.2017.1375130"},{"key":"e_1_2_1_34_1","volume-title":"How High -Tech Tools Profile, Police, and Punish the Poor","author":"Eubanks Virginia","unstructured":"Virginia Eubanks. 2018. Automating Inequality : How High -Tech Tools Profile, Police, and Punish the Poor .St. Martin's Press, New York."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025837"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.5555\/1866696.1866709"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.aos.2013.11.002"},{"key":"e_1_2_1_38_1","volume-title":"Hanna Wallach, Hal Daume\u00e9 III, and Kate Crawford.","author":"Gebru Timnit","year":"2018","unstructured":"Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daume\u00e9 III, and Kate Crawford. 2018. Datasheets for Datasets. arXiv:1803.09010 (March 2018). arxiv: 1803.09010"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372862"},{"key":"e_1_2_1_40_1","volume-title":"Measuring Social Biases of Crowd Workers Using Counterfactual Queries. In Workshop on Fair & Responsible AI at ACM CHI Conference on Human Factors in Computing Systems","author":"Ghai Bhavya","year":"2020","unstructured":"Bhavya Ghai, Q. Vera Liao, Yunfeng Zhang, and Klaus Mueller. 2020. Measuring Social Biases of Crowd Workers Using Counterfactual Queries. In Workshop on Fair & Responsible AI at ACM CHI Conference on Human Factors in Computing Systems. Honolulu, HI, USA."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/9780262525374.003.0009"},{"key":"e_1_2_1_42_1","doi-asserted-by":"crossref","unstructured":"Lisa Gitelman (Ed.). 2013. \"Raw Data\" Is an Oxymoron .The MIT Press Cambridge Massachusetts ; London England. Q355 .R385 2013","DOI":"10.7551\/mitpress\/9302.001.0001"},{"key":"e_1_2_1_43_1","volume-title":"Strauss","author":"Glaser Barney G.","year":"1998","unstructured":"Barney G. Glaser and Anselm L. Strauss. 1998. Grounded theory: Strategien qualitativer Forschung. Huber, Bern."},{"key":"e_1_2_1_44_1","volume-title":"Gray and Siddharth Suri","author":"Mary","year":"2019","unstructured":"Mary L. Gray and Siddharth Suri. 2019. Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass. Houghton Mifflin Harcourt, Boston."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173582"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372826"},{"key":"e_1_2_1_47_1","volume-title":"The Dataset Nutrition Label : A Framework To Drive Higher Data Quality Standards. arXiv:1805.03677","author":"Holland Sarah","year":"2018","unstructured":"Sarah Holland, Ahmed Hosny, Sarah Newman, Joshua Joseph, and Kasia Chmielinski. 2018. The Dataset Nutrition Label : A Framework To Drive Higher Data Quality Standards. arXiv:1805.03677 (2018)."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300637"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3375627.3375809"},{"key":"e_1_2_1_50_1","volume-title":"Designing for Service : Key Issues and New Directions","author":"Kimbell Lucy","unstructured":"Lucy Kimbell and Jeanette Blomberg. 2017. The Object of Service Design. In Designing for Service : Key Issues and New Directions. Bloomsbury Publishing, 81--94."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1080\/1369118X.2016.1154087"},{"key":"e_1_2_1_52_1","volume-title":"Peluso","author":"Klein Gary","year":"2007","unstructured":"Gary Klein, Jennifer K. Phillips, Erica L. Rall, and Deborah A. Peluso. 2007. A Data-Frame Theory of Sensemaking. In Expertise out of Context: Proceedings of the Sixth International Conference on Naturalistic Decision Making. Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US, 113--155."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1177\/1461444818779750"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.17516\/1997--1370--2015--8--1--114--129"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376445"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1080\/1369118X.2012.676056"},{"key":"e_1_2_1_57_1","volume-title":"The Metric Society: On the Quantification of the Social","author":"Mau Steffen","unstructured":"Steffen Mau. 2019. The Metric Society: On the Quantification of the Social .Polity, Cambridge ; Medford, MA. MR 2800 M447 M5"},{"key":"e_1_2_1_58_1","doi-asserted-by":"crossref","unstructured":"Frauke M\u00f6rike. 2019. Ethnography for Human Factors Researchers. Collecting and Interweaving Threads of HCI.","DOI":"10.1145\/3290607.3298814"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1007\/978--1--4939-0378--8_2"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/2957276.2957280"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300356"},{"key":"e_1_2_1_62_1","volume-title":"Algorithms of Oppression : How Search Engines Reinforce Racism","author":"Noble Safiya Umoja","unstructured":"Safiya Umoja Noble. 2018. Algorithms of Oppression : How Search Engines Reinforce Racism .NYU Press, New York."},{"key":"e_1_2_1_63_1","volume-title":"Weapons of Math Destruction : How Big Data Increases Inequality and Threatens Democracy .PENGUIN BOOKS","author":"O'Neil Cathy","unstructured":"Cathy O'Neil. 2017. Weapons of Math Destruction : How Big Data Increases Inequality and Threatens Democracy .PENGUIN BOOKS, London."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376780"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287567"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998331"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274405"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702298"},{"key":"e_1_2_1_69_1","volume-title":"Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program ). arXiv:2003.12206 (April","author":"Pineau Joelle","year":"2020","unstructured":"Joelle Pineau, Philippe Vincent-Lamarre, Koustuv Sinha, Vincent Larivi\u00e8re, Alina Beygelzimer, Florence d'Alch\u00e9-Buc, Emily Fox, and Hugo Larochelle. 2020. Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program ). arXiv:2003.12206 (April 2020). arxiv: 2003.12206"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.2543507"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359246"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/3392866"},{"key":"e_1_2_1_73_1","volume-title":"Knowing Algorithms. In digitalSTS : A Field Guide for Science & Technology Studies","author":"Seaver Nick","unstructured":"Nick Seaver. 2019. Knowing Algorithms. In digitalSTS : A Field Guide for Science & Technology Studies. Princeton University Press, PRINCETON; OXFORD, 412--422."},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.13140\/RG.2.2.36360.93448"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008651105359"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1080\/00313831.2011.581686"},{"key":"e_1_2_1_77_1","volume-title":"Proceedings of the 24th International Conference on Neural Information Processing Systems (NIPS '11)","author":"Fabian","year":"1800","unstructured":"Fabian L. Wauthier and Michael I. Jordan. 2011. Bayesian Bias Mitigation for Crowdsourcing. In Proceedings of the 24th International Conference on Neural Information Processing Systems (NIPS '11). Curran Associates Inc., Granada, Spain, 1800--1808."},{"key":"e_1_2_1_78_1","unstructured":"Jennifer Wortman Vaughan and Hanna Wallach. 2020. A Human -Centered Agenda for Intelligible Machine Learning. In Machines We Trust : Getting Along with Artificial Intelligence."},{"key":"e_1_2_1_79_1","volume-title":"The Fine Line : Making Distinctions in Everyday Life","author":"Zerubavel Eviatar","unstructured":"Eviatar Zerubavel. 1993. The Fine Line : Making Distinctions in Everyday Life. 2nd ed. ed.). University of Chicago Press.","edition":"2"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1145\/2684822.2685301"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.4324\/9781003320609-27"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3415186","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3415186","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T01:49:16Z","timestamp":1778550556000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3415186"}},"subtitle":["Power Dynamics in Data Annotation for Computer Vision"],"short-title":[],"issued":{"date-parts":[[2020,10,14]]},"references-count":81,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2020,10,14]]}},"alternative-id":["10.1145\/3415186"],"URL":"https:\/\/doi.org\/10.1145\/3415186","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,14]]},"assertion":[{"value":"2020-10-15","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}