{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T14:27:00Z","timestamp":1777040820888,"version":"3.51.4"},"reference-count":109,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW3","license":[{"start":{"date-parts":[[2021,1,5]],"date-time":"2021-01-05T00:00:00Z","timestamp":1609804800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100010661","name":"Horizon 2020 Framework Programme","doi-asserted-by":"publisher","award":["739578, 810105"],"award-info":[{"award-number":["739578, 810105"]}],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Government of the Republic of Cyprus"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,1,5]]},"abstract":"<jats:p>Machine-learned computer vision algorithms for tagging images are increasingly used by developers and researchers, having become popularized as easy-to-use \"cognitive services.\" Yet these tools struggle with gender recognition, particularly when processing images of women, people of color and non-binary individuals. Socio-technical researchers have cited data bias as a key problem; training datasets often over-represent images of people and contexts that convey social stereotypes. The social psychology literature explains that people learn social stereotypes, in part, by observing others in particular roles and contexts, and can inadvertently learn to associate gender with scenes, occupations and activities. Thus, we study the extent to which image tagging algorithms mimic this phenomenon. We design a controlled experiment, to examine the interdependence between algorithmic recognition of context and the depicted person's gender. In the spirit of auditing to understand machine behaviors, we create a highly controlled dataset of people images, imposed on gender-stereotyped backgrounds. Our methodology is reproducible and our code publicly available. Evaluating five proprietary algorithms, we find that in three, gender inference is hindered when a background is introduced. Of the two that \"see\" both backgrounds and gender, it is the one whose output is most consistent with human stereotyping processes that is superior in recognizing gender. We discuss the accuracy--fairness trade-off, as well as the importance of auditing black boxes in better understanding this double-edged sword.<\/jats:p>","DOI":"10.1145\/3432931","type":"journal-article","created":{"date-parts":[[2021,1,5]],"date-time":"2021-01-05T18:43:37Z","timestamp":1609872217000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":36,"title":["To \"See\" is to Stereotype"],"prefix":"10.1145","volume":"4","author":[{"given":"Pinar","family":"Barlas","sequence":"first","affiliation":[{"name":"Research Centre on Interactive Media, Smart Systems and Emerging Technologies, Nicosia, Cyprus"}]},{"given":"Kyriakos","family":"Kyriakou","sequence":"additional","affiliation":[{"name":"Research Centre on Interactive Media, Smart Systems and Emerging Technologies, Nicosia, Cyprus"}]},{"given":"Olivia","family":"Guest","sequence":"additional","affiliation":[{"name":"Research Centre on Interactive Media, Smart Systems and Emerging Technologies, Nicosia, Cyprus"}]},{"given":"Styliani","family":"Kleanthous","sequence":"additional","affiliation":[{"name":"Open University of Cyprus &amp; Research Centre on Interactive Media, Smart Systems and Emerging Technologies - RISE Ltd, Latsia, Cyprus"}]},{"given":"Jahna","family":"Otterbacher","sequence":"additional","affiliation":[{"name":"Open University of Cyprus &amp; Research Centre on Interactive Media, Smart Systems and Emerging Technologies - RISE Ltd, Latsia, Cyprus"}]}],"member":"320","published-online":{"date-parts":[[2021,1,5]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Role transitions","author":"Allen Vernon L","unstructured":"Vernon L Allen and Evert Van de Vliert . 1984. A role theoretical perspective on transitional processes . In Role transitions . Springer , 3--18. Vernon L Allen and Evert Van de Vliert. 1984. A role theoretical perspective on transitional processes. In Role transitions. Springer, 3--18."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819927"},{"key":"e_1_2_1_3_1","volume-title":"ProPublica","volume":"23","author":"Angwin Julia","year":"2016","unstructured":"Julia Angwin , Jeff Larson , Surya Mattu , and Lauren Kirchner . 2016 . Machine bias . ProPublica , May , Vol. 23 (2016), 2016. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica, May, Vol. 23 (2016), 2016."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1177\/107769900508200405"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the International AAAI Conference on Web and Social Media","volume":"13","author":"Bakhshi Saeideh","year":"2019","unstructured":"Saeideh Bakhshi , Lyndon Kennedy , Eric Gilbert , and David A Shamma . 2019 . Filtered Food and Nofilter Landscapes in Online Photography: The Role of Content and Visual Effects in Photo Engagement . In Proceedings of the International AAAI Conference on Web and Social Media , Vol. 13 . 80--90. Saeideh Bakhshi, Lyndon Kennedy, Eric Gilbert, and David A Shamma. 2019. Filtered Food and Nofilter Landscapes in Online Photography: The Role of Content and Visual Effects in Photo Engagement. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 13. 80--90."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10707-013-0197-8"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the International AAAI Conference on Web and Social Media","volume":"13","author":"Barlas Pinar","year":"2019","unstructured":"Pinar Barlas , Kyriakos Kyriakou , Styliani Kleanthous , and Jahna Otterbacher . 2019 . Social b (eye) as: Human and machine descriptions of people images . In Proceedings of the International AAAI Conference on Web and Social Media , Vol. 13 . 583--591. Pinar Barlas, Kyriakos Kyriakou, Styliani Kleanthous, and Jahna Otterbacher. 2019. Social b (eye) as: Human and machine descriptions of people images. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 13. 583--591."},{"key":"e_1_2_1_8_1","volume-title":"Mechanisms of linguistic bias: How words reflect and maintain stereotypic expectancies. Social Cognition and Communication","author":"Beukeboom CJ","year":"2014","unstructured":"CJ Beukeboom , JP Forgas , O Vincze , and J Laszlo . 2014. Mechanisms of linguistic bias: How words reflect and maintain stereotypic expectancies. Social Cognition and Communication ( 2014 ), 313--330. CJ Beukeboom, JP Forgas, O Vincze, and J Laszlo. 2014. Mechanisms of linguistic bias: How words reflect and maintain stereotypic expectancies. Social Cognition and Communication (2014), 313--330."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372864"},{"key":"e_1_2_1_10_1","volume-title":"Algorithmic Injustices: Towards a Relational Ethics. arXiv preprint arXiv:1912.07376","author":"Birhane Abeba","year":"2019","unstructured":"Abeba Birhane and Fred Cummins . 2019 . Algorithmic Injustices: Towards a Relational Ethics. arXiv preprint arXiv:1912.07376 (2019). Abeba Birhane and Fred Cummins. 2019. Algorithmic Injustices: Towards a Relational Ethics. arXiv preprint arXiv:1912.07376 (2019)."},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the International AAAI Conference on Web and Social Media","volume":"13","author":"Blandfort Philipp","year":"2019","unstructured":"Philipp Blandfort , Desmond U Patton , William R Frey , Svebor Karaman , Surabhi Bhargava , Fei-Tzin Lee , Siddharth Varia , Chris Kedzie , Michael B Gaskell , Rossano Schifanella , 2019 . Multimodal social media analysis for gang violence prevention . In Proceedings of the International AAAI Conference on Web and Social Media , Vol. 13 . 114--124. Philipp Blandfort, Desmond U Patton, William R Frey, Svebor Karaman, Surabhi Bhargava, Fei-Tzin Lee, Siddharth Varia, Chris Kedzie, Michael B Gaskell, Rossano Schifanella, et almbox. 2019. Multimodal social media analysis for gang violence prevention. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 13. 114--124."},{"key":"e_1_2_1_12_1","volume-title":"Affect, cognition and stereotyping","author":"Bodenhausen Galen V","unstructured":"Galen V Bodenhausen . 1993. Emotions , arousal, and stereotypic judgments: A heuristic model of affect and stereotyping . In Affect, cognition and stereotyping . Elsevier , 13--37. Galen V Bodenhausen. 1993. Emotions, arousal, and stereotypic judgments: A heuristic model of affect and stereotyping. In Affect, cognition and stereotyping. Elsevier, 13--37."},{"key":"e_1_2_1_13_1","unstructured":"Tolga Bolukbasi Kai-Wei Chang James Y Zou Venkatesh Saligrama and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems. 4349--4357.  Tolga Bolukbasi Kai-Wei Chang James Y Zou Venkatesh Saligrama and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems. 4349--4357."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1080\/03004430.2016.1223072"},{"key":"e_1_2_1_15_1","volume-title":"Conference on Fairness, Accountability and Transparency. 77--91","author":"Buolamwini Joy","year":"2018","unstructured":"Joy Buolamwini and Timnit Gebru . 2018 . Gender shades: Intersectional accuracy disparities in commercial gender classification . In Conference on Fairness, Accountability and Transparency. 77--91 . Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency. 77--91."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951715622512"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2019.8818751"},{"key":"e_1_2_1_18_1","volume-title":"Gender-stereotypic images of occupations correspond to the sex segregation of employment. Personality and social psychology bulletin","author":"Cejka Mary Ann","year":"1999","unstructured":"Mary Ann Cejka and Alice H Eagly . 1999. Gender-stereotypic images of occupations correspond to the sex segregation of employment. Personality and social psychology bulletin , Vol. 25 , 4 ( 1999 ), 413--423. Mary Ann Cejka and Alice H Eagly. 1999. Gender-stereotypic images of occupations correspond to the sex segregation of employment. Personality and social psychology bulletin, Vol. 25, 4 (1999), 413--423."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00287463"},{"key":"e_1_2_1_20_1","volume-title":"The New York Times [Internet]","volume":"883","author":"Chinoy Sahil","year":"2019","unstructured":"Sahil Chinoy . 2019 . The Racist History Behind Facial Recognition . The New York Times [Internet] , Vol. 883 (2019). Sahil Chinoy. 2019. The Racist History Behind Facial Recognition. The New York Times [Internet], Vol. 883 (2019)."},{"key":"e_1_2_1_21_1","volume-title":"The perpetuation of subtle prejudice: Race and gender imagery in 1990s television advertising. Sex roles","author":"Coltrane Scott","year":"2000","unstructured":"Scott Coltrane and Melinda Messineo . 2000. The perpetuation of subtle prejudice: Race and gender imagery in 1990s television advertising. Sex roles , Vol. 42 , 5--6 ( 2000 ), 363--389. Scott Coltrane and Melinda Messineo. 2000. The perpetuation of subtle prejudice: Race and gender imagery in 1990s television advertising. Sex roles, Vol. 42, 5--6 (2000), 363--389."},{"key":"e_1_2_1_22_1","article-title":"Design Justice, A.I., and Escape from the Matrix of Domination","volume":"7","author":"Costanza-Chock Sasha","year":"2018","unstructured":"Sasha Costanza-Chock . 2018 . Design Justice, A.I., and Escape from the Matrix of Domination . Journal of Design and Science (16 7 2018). https:\/\/doi.org\/10.21428\/96c8d426 https:\/\/jods.mitpress.mit.edu\/pub\/costanza-chock. 10.21428\/96c8d426 Sasha Costanza-Chock. 2018. Design Justice, A.I., and Escape from the Matrix of Domination. Journal of Design and Science (16 7 2018). https:\/\/doi.org\/10.21428\/96c8d426 https:\/\/jods.mitpress.mit.edu\/pub\/costanza-chock.","journal-title":"Journal of Design and Science (16"},{"key":"e_1_2_1_23_1","volume-title":"Warmth and competence as universal dimensions of social perception: The stereotype content model and the BIAS map. Advances in experimental social psychology","author":"Cuddy Amy JC","year":"2008","unstructured":"Amy JC Cuddy , Susan T Fiske , and Peter Glick . 2008. Warmth and competence as universal dimensions of social perception: The stereotype content model and the BIAS map. Advances in experimental social psychology , Vol. 40 ( 2008 ), 61--149. Amy JC Cuddy, Susan T Fiske, and Peter Glick. 2008. Warmth and competence as universal dimensions of social perception: The stereotype content model and the BIAS map. Advances in experimental social psychology, Vol. 40 (2008), 61--149."},{"key":"e_1_2_1_24_1","volume-title":"The dynamics of warmth and competence judgments, and their outcomes in organizations. Research in organizational behavior","author":"Cuddy Amy JC","year":"2011","unstructured":"Amy JC Cuddy , Peter Glick , and Anna Beninger . 2011. The dynamics of warmth and competence judgments, and their outcomes in organizations. Research in organizational behavior , Vol. 31 ( 2011 ), 73--98. Amy JC Cuddy, Peter Glick, and Anna Beninger. 2011. The dynamics of warmth and competence judgments, and their outcomes in organizations. Research in organizational behavior, Vol. 31 (2011), 73--98."},{"key":"e_1_2_1_25_1","doi-asserted-by":"crossref","unstructured":"David Danks and Alex John London. 2017. Algorithmic Bias in Autonomous Systems.. In IJCAI. 4691--4697.  David Danks and Alex John London. 2017. Algorithmic Bias in Autonomous Systems.. In IJCAI. 4691--4697.","DOI":"10.24963\/ijcai.2017\/654"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-11009-3_35"},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 52--59","author":"de Vries Terrance","unstructured":"Terrance de Vries , Ishan Misra , Changhan Wang , and Laurens van der Maaten. 2019. Does Object Recognition Work for Everyone? . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 52--59 . Terrance de Vries, Ishan Misra, Changhan Wang, and Laurens van der Maaten. 2019. Does Object Recognition Work for Everyone?. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 52--59."},{"key":"e_1_2_1_28_1","volume-title":"Eleventh International AAAI Conference on Web and Social Media.","author":"Deeb-Swihart Julia","year":"2017","unstructured":"Julia Deeb-Swihart , Christopher Polack , Eric Gilbert , and Irfan Essa . 2017 . Selfie-presentation in everyday life: A large-scale characterization of selfie contexts on instagram . In Eleventh International AAAI Conference on Web and Social Media. Julia Deeb-Swihart, Christopher Polack, Eric Gilbert, and Irfan Essa. 2017. Selfie-presentation in everyday life: A large-scale characterization of selfie contexts on instagram. In Eleventh International AAAI Conference on Web and Social Media."},{"key":"e_1_2_1_29_1","volume-title":"Microsoft Computer Vision APIs Distilled","author":"Sole Alessandro Del","unstructured":"Alessandro Del Sole . 2018. Introducing microsoft cognitive services . In Microsoft Computer Vision APIs Distilled . Springer , 1--4. Alessandro Del Sole. 2018. Introducing microsoft cognitive services. In Microsoft Computer Vision APIs Distilled. Springer, 1--4."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2844110"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025739"},{"key":"e_1_2_1_32_1","volume-title":"Conference on Fairness, Accountability and Transparency. 119--133","author":"Dwork Cynthia","year":"2018","unstructured":"Cynthia Dwork , Nicole Immorlica , Adam Tauman Kalai , and Max Leiserson . 2018 . Decoupled classifiers for group-fair and efficient machine learning . In Conference on Fairness, Accountability and Transparency. 119--133 . Cynthia Dwork, Nicole Immorlica, Adam Tauman Kalai, and Max Leiserson. 2018. Decoupled classifiers for group-fair and efficient machine learning. In Conference on Fairness, Accountability and Transparency. 119--133."},{"key":"e_1_2_1_33_1","volume-title":"Are people prejudiced against women? Some answers from research on attitudes, gender stereotypes, and judgments of competence. European review of social psychology","author":"Eagly Alice H","year":"1994","unstructured":"Alice H Eagly and Antonio Mladinic . 1994. Are people prejudiced against women? Some answers from research on attitudes, gender stereotypes, and judgments of competence. European review of social psychology , Vol. 5 , 1 ( 1994 ), 1--35. Alice H Eagly and Antonio Mladinic. 1994. Are people prejudiced against women? Some answers from research on attitudes, gender stereotypes, and judgments of competence. European review of social psychology, Vol. 5, 1 (1994), 1--35."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.43.5.915"},{"key":"e_1_2_1_35_1","first-page":"458","article-title":"Social Role Theory. In Handbook of theories of social psychology, John C Turner and Katherine J Reynolds (Eds.). Vol. 2. Sage London","volume":"49","author":"Eagly Alice H.","year":"2012","unstructured":"Alice H. Eagly and Wendy Wood . 2012 . Social Role Theory. In Handbook of theories of social psychology, John C Turner and Katherine J Reynolds (Eds.). Vol. 2. Sage London , Chapter 49 , 458 -- 476 . https:\/\/doi.org\/10.4135\/9781446249222.n49 10.4135\/9781446249222.n49 Alice H. Eagly and Wendy Wood. 2012. Social Role Theory. In Handbook of theories of social psychology, John C Turner and Katherine J Reynolds (Eds.). Vol. 2. Sage London, Chapter 49, 458--476. https:\/\/doi.org\/10.4135\/9781446249222.n49","journal-title":"Chapter"},{"key":"e_1_2_1_36_1","volume-title":"Eleventh International AAAI Conference on Web and Social Media (ICWSM). 62--71","author":"Eslami Motahhare","year":"2017","unstructured":"Motahhare Eslami , Kristen Vaccaro , Karrie Karahalios , and Kevin Hamilton . 2017 . ?Be careful; things can be worse than they appear?: Understanding Biased Algorithms and Users? Behavior around Them in Rating Platforms . In Eleventh International AAAI Conference on Web and Social Media (ICWSM). 62--71 . Motahhare Eslami, Kristen Vaccaro, Karrie Karahalios, and Kevin Hamilton. 2017. ?Be careful; things can be worse than they appear?: Understanding Biased Algorithms and Users? Behavior around Them in Rating Platforms. In Eleventh International AAAI Conference on Web and Social Media (ICWSM). 62--71."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1521\/soco.1988.6.1.21"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1080\/1369118X.2017.1330418"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611974348.17"},{"key":"e_1_2_1_40_1","volume-title":"Amy JC Cuddy, and Peter Glick","author":"Fiske Susan T","year":"2007","unstructured":"Susan T Fiske , Amy JC Cuddy, and Peter Glick . 2007 . Universal dimensions of social cognition: Warmth and competence. Trends in cognitive sciences, Vol. 11 , 2 (2007), 77--83. Susan T Fiske, Amy JC Cuddy, and Peter Glick. 2007. Universal dimensions of social cognition: Warmth and competence. Trends in cognitive sciences, Vol. 11, 2 (2007), 77--83."},{"key":"e_1_2_1_41_1","volume-title":"Applications of heuristics and biases to social issues","author":"Garb Howard N","unstructured":"Howard N Garb . 1994. Cognitive heuristics and biases in personality assessment . In Applications of heuristics and biases to social issues . Springer , 73--90. Howard N Garb. 1994. Cognitive heuristics and biases in personality assessment. In Applications of heuristics and biases to social issues. Springer, 73--90."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858234"},{"key":"e_1_2_1_43_1","volume-title":"The development of gender stereotyping of adult occupations in elementary school children. Child Development","author":"Garrett CS","year":"1977","unstructured":"CS Garrett , P Lynne Ein , and Leslie Tremaine . 1977. The development of gender stereotyping of adult occupations in elementary school children. Child Development ( 1977 ), 507--512. CS Garrett, P Lynne Ein, and Leslie Tremaine. 1977. The development of gender stereotyping of adult occupations in elementary school children. Child Development (1977), 507--512."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.6028\/NIST.IR.8280"},{"key":"e_1_2_1_45_1","volume-title":"Proceedings of the International AAAI Conference on Web and Social Media","volume":"13","author":"Guntuku Sharath Chandra","year":"2019","unstructured":"Sharath Chandra Guntuku , Daniel Preotiuc-Pietro , Johannes C Eichstaedt , and Lyle H Ungar . 2019 . What twitter profile and posted images reveal about depression and anxiety . In Proceedings of the International AAAI Conference on Web and Social Media , Vol. 13 . 236--246. Sharath Chandra Guntuku, Daniel Preotiuc-Pietro, Johannes C Eichstaedt, and Lyle H Ungar. 2019. What twitter profile and posted images reveal about depression and anxiety. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 13. 236--246."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173582"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998327"},{"key":"e_1_2_1_48_1","doi-asserted-by":"crossref","unstructured":"S Alexander Haslam John C Turner Penelope J Oakes Katherine J Reynolds and Bertjan Doosje. 2002. From personal pictures in the head to collective tools in the world: How shared stereotypes allow groups to represent and change social reality. (2002).  S Alexander Haslam John C Turner Penelope J Oakes Katherine J Reynolds and Bertjan Doosje. 2002. From personal pictures in the head to collective tools in the world: How shared stereotypes allow groups to represent and change social reality. (2002).","DOI":"10.1017\/CBO9780511489877.009"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1754-9434.2008.00072.x"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01219-9_47"},{"key":"e_1_2_1_51_1","volume-title":"International handbook of Internet research","author":"Herring Susan C","unstructured":"Susan C Herring . 2009. Web content analysis: Expanding the paradigm . In International handbook of Internet research . Springer , 233--249. Susan C Herring. 2009. Web content analysis: Expanding the paradigm. In International handbook of Internet research. Springer, 233--249."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1177\/1940161215596730"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v8i1.14578"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274342"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359300"},{"key":"e_1_2_1_56_1","volume-title":"Twelfth International AAAI Conference on Web and Social Media.","author":"Jung Soon-Gyo","year":"2018","unstructured":"Soon-Gyo Jung , Jisun An , Haewoon Kwak , Joni Salminen , and Bernard Jim Jansen . 2018 . Assessing the accuracy of four popular face recognition tools for inferring gender, age, and race . In Twelfth International AAAI Conference on Web and Social Media. Soon-Gyo Jung, Jisun An, Haewoon Kwak, Joni Salminen, and Bernard Jim Jansen. 2018. Assessing the accuracy of four popular face recognition tools for inferring gender, age, and race. In Twelfth International AAAI Conference on Web and Social Media."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702520"},{"key":"e_1_2_1_58_1","volume-title":"IBM Reseach","volume":"2","author":"Kelly John E","year":"2015","unstructured":"John E Kelly . 2015 . Computing, cognition and the future of knowing. Whitepaper , IBM Reseach , Vol. 2 (2015). John E Kelly. 2015. Computing, cognition and the future of knowing. Whitepaper, IBM Reseach, Vol. 2 (2015)."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274357"},{"key":"e_1_2_1_60_1","volume-title":"SPIT-IEEE Colloquium and International Conference","volume":"1","author":"Kinage Kishor S","year":"2008","unstructured":"Kishor S Kinage and SG Bhirud . 2008 . Racial inconsistency in face recognition . In SPIT-IEEE Colloquium and International Conference , Vol. 1 . 78--81. Kishor S Kinage and SG Bhirud. 2008. Racial inconsistency in face recognition. In SPIT-IEEE Colloquium and International Conference, Vol. 1. 78--81."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1080\/07491409.1998.10162557"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2012.2214212"},{"key":"e_1_2_1_63_1","volume-title":"Proceedings of Innovations in Theoretical Computer Science (ITCS)","author":"Kleinberg Jon","year":"2016","unstructured":"Jon Kleinberg , Sendhil Mullainathan , and Manish Raghavan . 2016 . Inherent Trade-Offs in the Fair Determination of Risk Scores . In Proceedings of Innovations in Theoretical Computer Science (ITCS) , 2017. http:\/\/arxiv.org\/abs\/1609.05807 Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2016. Inherent Trade-Offs in the Fair Determination of Risk Scores. In Proceedings of Innovations in Theoretical Computer Science (ITCS), 2017. http:\/\/arxiv.org\/abs\/1609.05807"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/2815675.2815714"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01159-8_12"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998321"},{"key":"e_1_2_1_67_1","volume-title":"Proceedings of the International AAAI Conference on Web and Social Media","volume":"13","author":"Kyriakou Kyriakos","year":"2019","unstructured":"Kyriakos Kyriakou , Pinar Barlas , Styliani Kleanthous , and Jahna Otterbacher . 2019 . Fairness in proprietary image tagging algorithms: A cross-platform audit on people images . In Proceedings of the International AAAI Conference on Web and Social Media , Vol. 13 . 313--322. Kyriakos Kyriakou, Pinar Barlas, Styliani Kleanthous, and Jahna Otterbacher. 2019. Fairness in proprietary image tagging algorithms: A cross-platform audit on people images. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 13. 313--322."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1080\/08838150801991971"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2015.7301352"},{"key":"e_1_2_1_70_1","volume-title":"Tenth international AAAI conference on web and social media.","author":"Liu Leqi","year":"2016","unstructured":"Leqi Liu , Daniel Preotiuc-Pietro , Zahra Riahi Samani , Mohsen E Moghaddam , and Lyle Ungar . 2016 . Analyzing personality through social media profile picture choice . In Tenth international AAAI conference on web and social media. Leqi Liu, Daniel Preotiuc-Pietro, Zahra Riahi Samani, Mohsen E Moghaddam, and Lyle Ungar. 2016. Analyzing personality through social media profile picture choice. In Tenth international AAAI conference on web and social media."},{"key":"e_1_2_1_71_1","volume-title":"The Chicago face database: A free stimulus set of faces and norming data. Behavior research methods","author":"Ma Debbie S","year":"2015","unstructured":"Debbie S Ma , Joshua Correll , and Bernd Wittenbrink . 2015. The Chicago face database: A free stimulus set of faces and norming data. Behavior research methods , Vol. 47 , 4 ( 2015 ), 1122--1135. Debbie S Ma, Joshua Correll, and Bernd Wittenbrink. 2015. The Chicago face database: A free stimulus set of faces and norming data. Behavior research methods, Vol. 47, 4 (2015), 1122--1135."},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.57.6.981"},{"key":"e_1_2_1_73_1","volume-title":"Wagner Meira Jr., and Virgilio Almeida.","author":"Magno Gabriel","year":"2016","unstructured":"Gabriel Magno , Camila Souza Araujo , Wagner Meira Jr., and Virgilio Almeida. 2016 . Stereotypes in Search Engine Results: Understanding The Role of Local and Global Factors . arXiv:1609.05413 [cs] (Sept. 2016). http:\/\/arxiv.org\/abs\/1609.05413 arXiv: 1609.05413. Gabriel Magno, Camila Souza Araujo, Wagner Meira Jr., and Virgilio Almeida. 2016. Stereotypes in Search Engine Results: Understanding The Role of Local and Global Factors. arXiv:1609.05413 [cs] (Sept. 2016). http:\/\/arxiv.org\/abs\/1609.05413 arXiv: 1609.05413."},{"key":"e_1_2_1_74_1","volume-title":"Conference on Fairness, Accountability and Transparency. 107--118","author":"Menon Aditya Krishna","year":"2018","unstructured":"Aditya Krishna Menon and Robert C Williamson . 2018 . The cost of fairness in binary classification . In Conference on Fairness, Accountability and Transparency. 107--118 . Aditya Krishna Menon and Robert C Williamson. 2018. The cost of fairness in binary classification. In Conference on Fairness, Accountability and Transparency. 107--118."},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1080\/10646170701309890"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2004.05.005"},{"key":"e_1_2_1_77_1","volume-title":"Understanding unequal gender classification accuracy from face images. arXiv preprint arXiv:1812.00099","author":"Muthukumar Vidya","year":"2018","unstructured":"Vidya Muthukumar , Tejaswini Pedapati , Nalini Ratha , Prasanna Sattigeri , Chai-Wah Wu , Brian Kingsbury , Abhishek Kumar , Samuel Thomas , Aleksandra Mojsilovic , and Kush R Varshney . 2018. Understanding unequal gender classification accuracy from face images. arXiv preprint arXiv:1812.00099 ( 2018 ). Vidya Muthukumar, Tejaswini Pedapati, Nalini Ratha, Prasanna Sattigeri, Chai-Wah Wu, Brian Kingsbury, Abhishek Kumar, Samuel Thomas, Aleksandra Mojsilovic, and Kush R Varshney. 2018. Understanding unequal gender classification accuracy from face images. arXiv preprint arXiv:1812.00099 (2018)."},{"key":"e_1_2_1_78_1","volume-title":"Proc. Conf. Fairness Accountability Transp.","author":"Narayanan Arvind","year":"2018","unstructured":"Arvind Narayanan . 2018 . Translation tutorial: 21 fairness definitions and their politics . In Proc. Conf. Fairness Accountability Transp. , New York, USA. Arvind Narayanan. 2018. Translation tutorial: 21 fairness definitions and their politics. In Proc. Conf. Fairness Accountability Transp., New York, USA."},{"key":"e_1_2_1_79_1","volume-title":"Alan Hartman, and Tsvi Kuflik.","author":"Orphanou Kalia","year":"2020","unstructured":"Kalia Orphanou , Jahna Otterbacher , Styliani Kleanthous , Khuyagbaatar Batsuren , Fausto Giunchiglia , Veronika Bogina , Avital Shulner Tal , Alan Hartman, and Tsvi Kuflik. 2020 . Deliverable D3.4 - Survey Article. http:\/\/www.cycat.io\/wp-content\/uploads\/2020\/06\/D3.4_Survey_Article_NV.pdf Project deliverable, H2020 CyCAT ( 810105). Kalia Orphanou, Jahna Otterbacher, Styliani Kleanthous, Khuyagbaatar Batsuren, Fausto Giunchiglia, Veronika Bogina, Avital Shulner Tal, Alan Hartman, and Tsvi Kuflik. 2020. Deliverable D3.4 - Survey Article. http:\/\/www.cycat.io\/wp-content\/uploads\/2020\/06\/D3.4_Survey_Article_NV.pdf Project deliverable, H2020 CyCAT (810105)."},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702151"},{"key":"e_1_2_1_81_1","volume-title":"Group (Un) Fairness in Natural Language Image Descriptions. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing","volume":"7","author":"Otterbacher Jahna","year":"2019","unstructured":"Jahna Otterbacher , Pinar Barlas , Styliani Kleanthous , and Kyriakos Kyriakou . 2019 . How Do We Talk about Other People ? Group (Un) Fairness in Natural Language Image Descriptions. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing , Vol. 7 . 106--114. Jahna Otterbacher, Pinar Barlas, Styliani Kleanthous, and Kyriakos Kyriakou. 2019. How Do We Talk about Other People? Group (Un) Fairness in Natural Language Image Descriptions. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. 106--114."},{"key":"e_1_2_1_82_1","volume-title":"Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. https:\/\/doi.org\/10","author":"Otterbacher J.","unstructured":"J. Otterbacher , J. Bates , and P. D. Clough . 2017. Competent Men and Warm Women: Gender Stereotypes and Backlash in Image Search Results . In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. https:\/\/doi.org\/10 .1145\/3025453.3025727 10.1145\/3025453.3025727 J. Otterbacher, J. Bates, and P. D. Clough. 2017. Competent Men and Warm Women: Gender Stereotypes and Backlash in Image Search Results. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. https:\/\/doi.org\/10.1145\/3025453.3025727"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compeleceng.2017.11.022"},{"key":"e_1_2_1_84_1","volume-title":"One-step and two-step classification for abusive language detection on twitter. arXiv preprint arXiv:1706.01206","author":"Park Ji Ho","year":"2017","unstructured":"Ji Ho Park and Pascale Fung . 2017. One-step and two-step classification for abusive language detection on twitter. arXiv preprint arXiv:1706.01206 ( 2017 ). Ji Ho Park and Pascale Fung. 2017. One-step and two-step classification for abusive language detection on twitter. arXiv preprint arXiv:1706.01206 (2017)."},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357160"},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611972795.50"},{"key":"e_1_2_1_87_1","unstructured":"David Martin Powers. 2011. Evaluation: from precision recall and F-measure to ROC informedness markedness and correlation. (2011).  David Martin Powers. 2011. Evaluation: from precision recall and F-measure to ROC informedness markedness and correlation. (2011)."},{"key":"e_1_2_1_88_1","volume-title":"Large image datasets: A pyrrhic win for computer vision?arxiv","author":"Prabhu Vinay Uday","year":"2006","unstructured":"Vinay Uday Prabhu and Abeba Birhane . 2020. Large image datasets: A pyrrhic win for computer vision?arxiv : 2006 .16923 [cs.CY] Vinay Uday Prabhu and Abeba Birhane. 2020. Large image datasets: A pyrrhic win for computer vision?arxiv: 2006.16923 [cs.CY]"},{"key":"e_1_2_1_89_1","volume-title":"Cynthia Breazeal, Jacob W Crandall, Nicholas A Christakis, Iain D Couzin, Matthew O Jackson, et almbox.","author":"Rahwan Iyad","year":"2019","unstructured":"Iyad Rahwan , Manuel Cebrian , Nick Obradovich , Josh Bongard , Jean-Francc ois Bonnefon , Cynthia Breazeal, Jacob W Crandall, Nicholas A Christakis, Iain D Couzin, Matthew O Jackson, et almbox. 2019 . Machine behaviour. Nature , Vol. 568 , 7753 (2019), 477. Iyad Rahwan, Manuel Cebrian, Nick Obradovich, Josh Bongard, Jean-Francc ois Bonnefon, Cynthia Breazeal, Jacob W Crandall, Nicholas A Christakis, Iain D Couzin, Matthew O Jackson, et almbox. 2019. Machine behaviour. Nature, Vol. 568, 7753 (2019), 477."},{"key":"e_1_2_1_90_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-1609"},{"key":"e_1_2_1_91_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2018.08.032"},{"key":"e_1_2_1_92_1","volume-title":"Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: converting critical concerns into productive inquiry","author":"Sandvig Christian","year":"2014","unstructured":"Christian Sandvig , Kevin Hamilton , Karrie Karahalios , and Cedric Langbort . 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: converting critical concerns into productive inquiry ( 2014 ), 1--23. Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: converting critical concerns into productive inquiry (2014), 1--23."},{"key":"e_1_2_1_93_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359246"},{"key":"e_1_2_1_94_1","volume-title":"Effects of gender stereotypes and stereotype threat on children's performance on a spatial task. Revue internationale de psychologie sociale","author":"Shenouda Christine K","year":"2014","unstructured":"Christine K Shenouda and Judith H Danovitch . 2014. Effects of gender stereotypes and stereotype threat on children's performance on a spatial task. Revue internationale de psychologie sociale , Vol. 27 , 3 ( 2014 ), 53--77. Christine K Shenouda and Judith H Danovitch. 2014. Effects of gender stereotypes and stereotype threat on children's performance on a spatial task. Revue internationale de psychologie sociale, Vol. 27, 3 (2014), 53--77."},{"key":"e_1_2_1_95_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2018.2889231"},{"key":"e_1_2_1_96_1","doi-asserted-by":"publisher","DOI":"10.1016\/0001-8791(75)90037-8"},{"key":"e_1_2_1_97_1","volume-title":"Female Librarians and Male Computer Programmers? Gender Bias in Occupational Images on Digital Media Platforms. arXiv preprint arXiv:1912.05474","author":"Singh Vivek","year":"2019","unstructured":"Vivek Singh , Mary Chayko , Raj Inamdar , and Diana Floegel . 2019. Female Librarians and Male Computer Programmers? Gender Bias in Occupational Images on Digital Media Platforms. arXiv preprint arXiv:1912.05474 ( 2019 ). Vivek Singh, Mary Chayko, Raj Inamdar, and Diana Floegel. 2019. Female Librarians and Male Computer Programmers? Gender Bias in Occupational Images on Digital Media Platforms. arXiv preprint arXiv:1912.05474 (2019)."},{"key":"e_1_2_1_98_1","first-page":"35","article-title":"Modern information retrieval: A brief overview","volume":"24","author":"Amit Singhal","year":"2001","unstructured":"Amit Singhal et almbox. 2001 . Modern information retrieval: A brief overview . IEEE Data Eng. Bull. , Vol. 24 , 4 (2001), 35 -- 43 . Amit Singhal et almbox. 2001. Modern information retrieval: A brief overview. IEEE Data Eng. Bull., Vol. 24, 4 (2001), 35--43.","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_2_1_99_1","volume-title":"Facial recognition, emotion and race in animated social media. First Monday","author":"Stark Luke","year":"2018","unstructured":"Luke Stark . 2018. Facial recognition, emotion and race in animated social media. First Monday ( 2018 ). Luke Stark. 2018. Facial recognition, emotion and race in animated social media. First Monday (2018)."},{"key":"e_1_2_1_100_1","doi-asserted-by":"publisher","DOI":"10.1145\/2460276.2460278"},{"key":"e_1_2_1_101_1","volume-title":"Proceedings of the Workshop on Multimodal Corpora (MMC-2016)","author":"Miltenburg Emiel Van","year":"2016","unstructured":"Emiel Van Miltenburg . 2016 . Stereotyping and Bias in the Flickr30K Dataset . In Proceedings of the Workshop on Multimodal Corpora (MMC-2016) . 1--4. Emiel Van Miltenburg. 2016. Stereotyping and Bias in the Flickr30K Dataset. In Proceedings of the Workshop on Multimodal Corpora (MMC-2016). 1--4."},{"key":"e_1_2_1_102_1","doi-asserted-by":"publisher","DOI":"10.1145\/3194770.3194776"},{"key":"e_1_2_1_103_1","volume-title":"Ninth international AAAI conference on web and social media .","author":"Wagner Claudia","year":"2015","unstructured":"Claudia Wagner , David Garcia , Mohsen Jadidi , and Markus Strohmaier . 2015 . It's a man's Wikipedia? Assessing gender inequality in an online encyclopedia . In Ninth international AAAI conference on web and social media . Claudia Wagner, David Garcia, Mohsen Jadidi, and Markus Strohmaier. 2015. It's a man's Wikipedia? Assessing gender inequality in an online encyclopedia. In Ninth international AAAI conference on web and social media ."},{"key":"e_1_2_1_104_1","volume-title":"Implicit and explicit occupational gender stereotypes. Sex roles","author":"White Michael J","year":"2006","unstructured":"Michael J White and Gwendolen B White . 2006. Implicit and explicit occupational gender stereotypes. Sex roles , Vol. 55 , 3--4 ( 2006 ), 259--266. Michael J White and Gwendolen B White. 2006. Implicit and explicit occupational gender stereotypes. Sex roles, Vol. 55, 3--4 (2006), 259--266."},{"key":"e_1_2_1_105_1","volume-title":"Predictive inequity in object detection. arXiv preprint arXiv:1902.11097","author":"Wilson Benjamin","year":"2019","unstructured":"Benjamin Wilson , Judy Hoffman , and Jamie Morgenstern . 2019. Predictive inequity in object detection. arXiv preprint arXiv:1902.11097 ( 2019 ). Benjamin Wilson, Judy Hoffman, and Jamie Morgenstern. 2019. Predictive inequity in object detection. arXiv preprint arXiv:1902.11097 (2019)."},{"key":"e_1_2_1_106_1","doi-asserted-by":"publisher","DOI":"10.1080\/1369118X.2016.1200645"},{"key":"e_1_2_1_107_1","volume-title":"Prototypes are attractive because they are easy on the mind. Psychological science","author":"Winkielman Piotr","year":"2006","unstructured":"Piotr Winkielman , Jamin Halberstadt , Tedra Fazendeiro , and Steve Catty . 2006. Prototypes are attractive because they are easy on the mind. Psychological science , Vol. 17 , 9 ( 2006 ), 799--806. Piotr Winkielman, Jamin Halberstadt, Tedra Fazendeiro, and Steve Catty. 2006. Prototypes are attractive because they are easy on the mind. Psychological science, Vol. 17, 9 (2006), 799--806."},{"key":"e_1_2_1_108_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1323"},{"key":"e_1_2_1_109_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cognition.2004.12.003"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3432931","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3432931","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:47:12Z","timestamp":1750193232000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3432931"}},"subtitle":["Image Tagging Algorithms, Gender Recognition, and the Accuracy-Fairness Trade-off"],"short-title":[],"issued":{"date-parts":[[2021,1,5]]},"references-count":109,"journal-issue":{"issue":"CSCW3","published-print":{"date-parts":[[2021,1,5]]}},"alternative-id":["10.1145\/3432931"],"URL":"https:\/\/doi.org\/10.1145\/3432931","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,5]]},"assertion":[{"value":"2021-01-05","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}