{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,24]],"date-time":"2025-08-24T01:39:56Z","timestamp":1755999596763,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":37,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,6,26]],"date-time":"2022-06-26T00:00:00Z","timestamp":1656201600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"the ARC Discovery Project","award":["DP190102141"],"award-info":[{"award-number":["DP190102141"]}]},{"name":"the ARC Training Centre for Information Resilience","award":["IC200100022"],"award-info":[{"award-number":["IC200100022"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,6,26]]},"DOI":"10.1145\/3501247.3531588","type":"proceedings-article","created":{"date-parts":[[2022,6,24]],"date-time":"2022-06-24T22:41:09Z","timestamp":1656110469000},"page":"98-109","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Socio-Economic Diversity in Human Annotations"],"prefix":"10.1145","author":[{"given":"Shaoyang","family":"Fan","sequence":"first","affiliation":[{"name":"The University of Queensland, Australia"}]},{"given":"Pinar","family":"Barlas","sequence":"additional","affiliation":[{"name":"CYENS Centre of Excellence, Cyprus"}]},{"given":"Evgenia","family":"Christoforou","sequence":"additional","affiliation":[{"name":"CYENS Centre of Excellence, Cyprus"}]},{"given":"Jahna","family":"Otterbacher","sequence":"additional","affiliation":[{"name":"Open University of Cyprus, Cyprus and CYENS CoE, Cyprus"}]},{"given":"Shazia","family":"Sadiq","sequence":"additional","affiliation":[{"name":"The University of Queensland, Australia"}]},{"given":"Gianluca","family":"Demartini","sequence":"additional","affiliation":[{"name":"The University of Queensland, Australia"}]}],"member":"320","published-online":{"date-parts":[[2022,6,26]]},"reference":[{"key":"e_1_3_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240772"},{"key":"e_1_3_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1177\/016555159702300403"},{"key":"e_1_3_2_2_3_1","volume-title":"Accuracy-Fairness Trade-off. Proceedings of the ACM on Human-Computer Interaction 4, CSCW3(2021)","author":"Barlas Pinar","year":"2021","unstructured":"Pinar Barlas , Kyriakos Kyriakou , Olivia Guest , Styliani Kleanthous , and Jahna Otterbacher . 2021 . To \u201dSee\u201d is to Stereotype: Image Tagging Algorithms, Gender Recognition, and the Accuracy-Fairness Trade-off. Proceedings of the ACM on Human-Computer Interaction 4, CSCW3(2021) , 1\u201331. Pinar Barlas, Kyriakos Kyriakou, Olivia Guest, Styliani Kleanthous, and Jahna Otterbacher. 2021. To \u201dSee\u201d is to Stereotype: Image Tagging Algorithms, Gender Recognition, and the Accuracy-Fairness Trade-off. Proceedings of the ACM on Human-Computer Interaction 4, CSCW3(2021), 1\u201331."},{"key":"e_1_3_2_2_4_1","volume-title":"Proceedings of ICWSM, Vol.\u00a013","author":"Barlas P\u0131nar","year":"2019","unstructured":"P\u0131nar Barlas , Kyriakos Kyriakou , Styliani Kleanthous , and Jahna Otterbacher . 2019 . Social b (eye) as: Human and machine descriptions of people images . In Proceedings of ICWSM, Vol.\u00a013 . 583\u2013591. P\u0131nar Barlas, Kyriakos Kyriakou, Styliani Kleanthous, and Jahna Otterbacher. 2019. Social b (eye) as: Human and machine descriptions of people images. In Proceedings of ICWSM, Vol.\u00a013. 583\u2013591."},{"key":"e_1_3_2_2_5_1","unstructured":"Sebastian Chan. 2007. Tagging and Searching\u2013Serendipity and museum collection databases. In Museums and the Web Vol.\u00a02007. 87\u201399.  Sebastian Chan. 2007. Tagging and Searching\u2013Serendipity and museum collection databases. In Museums and the Web Vol.\u00a02007. 87\u201399."},{"key":"e_1_3_2_2_6_1","volume-title":"Sixth AAAI Conference on Human Computation and Crowdsourcing.","author":"Checco Alessandro","year":"2018","unstructured":"Alessandro Checco , Jo Bates , and Gianluca Demartini . 2018 . All that glitters is gold\u2014An attack scheme on gold questions in crowdsourcing . In Sixth AAAI Conference on Human Computation and Crowdsourcing. Alessandro Checco, Jo Bates, and Gianluca Demartini. 2018. All that glitters is gold\u2014An attack scheme on gold questions in crowdsourcing. In Sixth AAAI Conference on Human Computation and Crowdsourcing."},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02289138"},{"key":"e_1_3_2_2_8_1","unstructured":"Amit Datta Michael\u00a0Carl Tschantz and Anupam Datta. 2014. Automated experiments on ad privacy settings: A tale of opacity choice and discrimination. arXiv preprint arXiv:1408.6491(2014).  Amit Datta Michael\u00a0Carl Tschantz and Anupam Datta. 2014. Automated experiments on ad privacy settings: A tale of opacity choice and discrimination. arXiv preprint arXiv:1408.6491(2014)."},{"key":"e_1_3_2_2_9_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops. 52\u201359","author":"De\u00a0Vries Terrance","year":"2019","unstructured":"Terrance De\u00a0Vries , Ishan Misra , Changhan Wang , and Laurens van\u00a0der Maaten . 2019 . Does object recognition work for everyone? . In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops. 52\u201359 . Terrance De\u00a0Vries, Ishan Misra, Changhan Wang, and Laurens van\u00a0der Maaten. 2019. Does object recognition work for everyone?. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops. 52\u201359."},{"key":"e_1_3_2_2_10_1","volume-title":"How social-class stereotypes maintain inequality. Current opinion in psychology 18","author":"Durante Federica","year":"2017","unstructured":"Federica Durante and Susan\u00a0 T Fiske . 2017. How social-class stereotypes maintain inequality. Current opinion in psychology 18 ( 2017 ), 43\u201348. Federica Durante and Susan\u00a0T Fiske. 2017. How social-class stereotypes maintain inequality. Current opinion in psychology 18 (2017), 43\u201348."},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145310"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23656"},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2012.2202676"},{"key":"e_1_3_2_2_14_1","unstructured":"Gapminder. 2021. Dollar Street - photos as data to kill country stereotypes. https:\/\/www.gapminder.org\/dollar-street  Gapminder. 2021. Dollar Street - photos as data to kill country stereotypes. https:\/\/www.gapminder.org\/dollar-street"},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"crossref","unstructured":"Riste Gligorov Michiel Hildebrand Jacco Van\u00a0Ossenbruggen Guus Schreiber and Lora Aroyo. 2011. On the role of user-generated metadata in audio visual collections. In The sixth international conference on Knowledge capture. 145\u2013152.  Riste Gligorov Michiel Hildebrand Jacco Van\u00a0Ossenbruggen Guus Schreiber and Lora Aroyo. 2011. On the role of user-generated metadata in audio visual collections. In The sixth international conference on Knowledge capture. 145\u2013152.","DOI":"10.1145\/1999676.1999702"},{"key":"e_1_3_2_2_16_1","unstructured":"Scott Golder and Bernardo\u00a0A Huberman. 2005. The structure of collaborative tagging systems. arXiv preprint cs\/0508082(2005).  Scott Golder and Bernardo\u00a0A Huberman. 2005. The structure of collaborative tagging systems. arXiv preprint cs\/0508082(2005)."},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2019.2948168"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2004.03.002"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3020165.3022143"},{"key":"e_1_3_2_2_20_1","volume-title":"Proceedings of the International AAAI Conference on Web and Social Media, Vol.\u00a08.","author":"Hutto Clayton","year":"2014","unstructured":"Clayton Hutto and Eric Gilbert . 2014 . Vader: A parsimonious rule-based model for sentiment analysis of social media text . In Proceedings of the International AAAI Conference on Web and Social Media, Vol.\u00a08. Clayton Hutto and Eric Gilbert. 2014. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the International AAAI Conference on Web and Social Media, Vol.\u00a08."},{"key":"e_1_3_2_2_21_1","volume-title":"An Experimental Study of Bias in Platform Worker Ratings: The Role of Performance Quality and Gender. In The 2020 CHI Conference on Human Factors in Computing Systems. 1\u201313","author":"Jahanbakhsh F","year":"2020","unstructured":"F Jahanbakhsh , J Cranshaw , S Counts , W\u00a0S Lasecki , and K Inkpen . 2020 . An Experimental Study of Bias in Platform Worker Ratings: The Role of Performance Quality and Gender. In The 2020 CHI Conference on Human Factors in Computing Systems. 1\u201313 . F Jahanbakhsh, J Cranshaw, S Counts, W\u00a0S Lasecki, and K Inkpen. 2020. An Experimental Study of Bias in Platform Worker Ratings: The Role of Performance Quality and Gender. In The 2020 CHI Conference on Human Factors in Computing Systems. 1\u201313."},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702520"},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.22950"},{"key":"e_1_3_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1177\/1745691616673192"},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445507"},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/s41542-017-0009-x"},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702151"},{"key":"e_1_3_2_2_28_1","volume-title":"Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol.\u00a07. 106\u2013114","author":"Otterbacher Jahna","year":"2019","unstructured":"Jahna Otterbacher , P\u0131nar Barlas , Styliani Kleanthous , and Kyriakos Kyriakou . 2019 . How do we talk about other people? group (un) fairness in natural language image descriptions . In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol.\u00a07. 106\u2013114 . Jahna Otterbacher, P\u0131nar Barlas, Styliani Kleanthous, and Kyriakos Kyriakou. 2019. How do we talk about other people? group (un) fairness in natural language image descriptions. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol.\u00a07. 106\u2013114."},{"key":"e_1_3_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025727"},{"key":"e_1_3_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210094"},{"volume-title":"Studies in iconology: humanistic themes in the art of the Renaissance","author":"Panofsky Erwin","key":"e_1_3_2_2_31_1","unstructured":"Erwin Panofsky . 2018. Studies in iconology: humanistic themes in the art of the Renaissance . Routledge . Erwin Panofsky. 2018. Studies in iconology: humanistic themes in the art of the Renaissance. Routledge."},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2019.2926996"},{"volume-title":"CHI \u201910 Extended Abstracts on Human Factors in Computing Systems(CHI EA \u201910)","author":"Ross Joel","key":"e_1_3_2_2_33_1","unstructured":"Joel Ross , Lilly Irani , M.\u00a0 Six Silberman , Andrew Zaldivar , and Bill Tomlinson . 2010. Who Are the Crowdworkers? Shifting Demographics in Mechanical Turk . In CHI \u201910 Extended Abstracts on Human Factors in Computing Systems(CHI EA \u201910) . Association for Computing Machinery , New York, NY, USA , 2863\u20132872. Joel Ross, Lilly Irani, M.\u00a0Six Silberman, Andrew Zaldivar, and Bill Tomlinson. 2010. Who Are the Crowdworkers? Shifting Demographics in Mechanical Turk. In CHI \u201910 Extended Abstracts on Human Factors in Computing Systems(CHI EA \u201910). Association for Computing Machinery, New York, NY, USA, 2863\u20132872."},{"key":"e_1_3_2_2_34_1","volume-title":"Analyzing the subject of a picture: a theoretical approach. Cataloging & classification quarterly 6, 3","author":"Shatford Sara","year":"1986","unstructured":"Sara Shatford . 1986. Analyzing the subject of a picture: a theoretical approach. Cataloging & classification quarterly 6, 3 ( 1986 ), 39\u201362. Sara Shatford. 1986. Analyzing the subject of a picture: a theoretical approach. Cataloging & classification quarterly 6, 3 (1986), 39\u201362."},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.5555\/3192790.3192806"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2333112.2333120"},{"key":"e_1_3_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1978963"}],"event":{"name":"WebSci '22: 14th ACM Web Science Conference 2022","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web"],"location":"Barcelona Spain","acronym":"WebSci '22"},"container-title":["14th ACM Web Science Conference 2022"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3501247.3531588","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3501247.3531588","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:12:18Z","timestamp":1750191138000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3501247.3531588"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,26]]},"references-count":37,"alternative-id":["10.1145\/3501247.3531588","10.1145\/3501247"],"URL":"https:\/\/doi.org\/10.1145\/3501247.3531588","relation":{},"subject":[],"published":{"date-parts":[[2022,6,26]]},"assertion":[{"value":"2022-06-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}