{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T14:20:21Z","timestamp":1775571621243,"version":"3.50.1"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2019,12,1]],"date-time":"2019-12-01T00:00:00Z","timestamp":1575158400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGIR Forum"],"published-print":{"date-parts":[[2019,12]]},"abstract":"<jats:p>The purpose of the SIGIR 2019 workshop on Fairness, Accountability, Confidentiality, Transparency, and Safety (FACTS-IR) was to explore challenges in responsible information retrieval system development and deployment. To this end, the workshop aimed to crowd-source from the larger SIGIR community and draft an actionable research agenda on five key dimensions of responsible information retrieval: fairness, accountability, confidentiality, transparency, and safety. Such an agenda can guide others in the community that are interested in pursuing FACTS-IR research, as well as inform potential funders about relevant research avenues. The workshop brought together a diverse set of researchers and practitioners interested in contributing to the development of a technical research agenda for responsible information retrieval.<\/jats:p>","DOI":"10.1145\/3458553.3458556","type":"journal-article","created":{"date-parts":[[2021,3,23]],"date-time":"2021-03-23T22:17:49Z","timestamp":1616537869000},"page":"20-43","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":29,"title":["FACTS-IR"],"prefix":"10.1145","volume":"53","author":[{"given":"Alexandra","family":"Olteanu","sequence":"first","affiliation":[]},{"given":"Jean","family":"Garcia-Gathright","sequence":"additional","affiliation":[]},{"given":"Maarten","family":"de Rijke","sequence":"additional","affiliation":[]},{"given":"Michael D.","family":"Ekstrand","sequence":"additional","affiliation":[]},{"given":"Adam","family":"Roegiest","sequence":"additional","affiliation":[]},{"given":"Aldo","family":"Lipani","sequence":"additional","affiliation":[]},{"given":"Alex","family":"Beutel","sequence":"additional","affiliation":[]},{"given":"Alexandra","family":"Olteanu","sequence":"additional","affiliation":[]},{"given":"Ana","family":"Lucic","sequence":"additional","affiliation":[]},{"given":"Ana-Andreea","family":"Stoica","sequence":"additional","affiliation":[]},{"given":"Anubrata","family":"Das","sequence":"additional","affiliation":[]},{"given":"Asia","family":"Biega","sequence":"additional","affiliation":[]},{"given":"Bart","family":"Voorn","sequence":"additional","affiliation":[]},{"given":"Claudia","family":"Hauff","sequence":"additional","affiliation":[]},{"given":"Damiano","family":"Spina","sequence":"additional","affiliation":[]},{"given":"David","family":"Lewis","sequence":"additional","affiliation":[]},{"given":"Douglas W.","family":"Oard","sequence":"additional","affiliation":[]},{"given":"Emine","family":"Yilmaz","sequence":"additional","affiliation":[]},{"given":"Faegheh","family":"Hasibi","sequence":"additional","affiliation":[]},{"given":"Gabriella","family":"Kazai","sequence":"additional","affiliation":[]},{"given":"Graham","family":"McDonald","sequence":"additional","affiliation":[]},{"given":"Hinda","family":"Haned","sequence":"additional","affiliation":[]},{"given":"Iadh","family":"Ounis","sequence":"additional","affiliation":[]},{"given":"Ilse","family":"van der Linden","sequence":"additional","affiliation":[]},{"given":"Jean","family":"Garcia-Gathright","sequence":"additional","affiliation":[]},{"given":"Joris","family":"Baan","sequence":"additional","affiliation":[]},{"given":"Kamuela N.","family":"Lau","sequence":"additional","affiliation":[]},{"given":"Krisztian","family":"Balog","sequence":"additional","affiliation":[]},{"given":"Maarten","family":"de Rijke","sequence":"additional","affiliation":[]},{"given":"Mahmoud","family":"Sayed","sequence":"additional","affiliation":[]},{"given":"Maria","family":"Panteli","sequence":"additional","affiliation":[]},{"given":"Mark","family":"Sanderson","sequence":"additional","affiliation":[]},{"given":"Matthew","family":"Lease","sequence":"additional","affiliation":[]},{"given":"Michael D.","family":"Ekstrand","sequence":"additional","affiliation":[]},{"given":"Preethi","family":"Lahoti","sequence":"additional","affiliation":[]},{"given":"Toshihiro","family":"Kamishima","sequence":"additional","affiliation":[]}],"member":"320","published-online":{"date-parts":[[2021,3,23]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274784.3274788"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209581"},{"key":"e_1_2_1_3_1","volume-title":"Big data's disparate impact. California Law Review, 104:671","author":"Barocas Solon","year":"2016","unstructured":"Solon Barocas and Andrew D Selbst . Big data's disparate impact. California Law Review, 104:671 , 2016 . Solon Barocas and Andrew D Selbst. Big data's disparate impact. California Law Review, 104:671, 2016."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210063"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the 6th AAAI Conference on Human Computation and Crowdsourcing (HCOMP): Works-in-Progress Track","author":"Dang Brandon","year":"2018","unstructured":"Brandon Dang , Martin J. Riedl , and Matthew Lease . But who protects the moderators? the case of crowdsourced image moderation . In Proceedings of the 6th AAAI Conference on Human Computation and Crowdsourcing (HCOMP): Works-in-Progress Track , 2018 . Brandon Dang, Martin J. Riedl, and Matthew Lease. But who protects the moderators? the case of crowdsourced image moderation. In Proceedings of the 6th AAAI Conference on Human Computation and Crowdsourcing (HCOMP): Works-in-Progress Track, 2018."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287572"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1037\/xge0000033"},{"key":"e_1_2_1_8_1","volume-title":"L119:1--88","author":"EU.","year":"2016","unstructured":"EU. Regulation (eu) 2016\/679 of the european parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95\/46\/ec (general data protection regulation). Official Journal of the European Union , L119:1--88 , 2016 . EU. Regulation (eu) 2016\/679 of the european parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95\/46\/ec (general data protection regulation). Official Journal of the European Union, L119:1--88, 2016."},{"key":"e_1_2_1_9_1","volume-title":"Filter bubbles, echo chambers, and online news consumption. Public Opinion Quarterly, 80(S1):298--320","author":"Flaxman Seth","year":"2016","unstructured":"Seth Flaxman , Sharad Goel , and Justin M Rao . Filter bubbles, echo chambers, and online news consumption. Public Opinion Quarterly, 80(S1):298--320 , 2016 . Seth Flaxman, Sharad Goel, and Justin M Rao. Filter bubbles, echo chambers, and online news consumption. Public Opinion Quarterly, 80(S1):298--320, 2016."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/230538.230561"},{"key":"e_1_2_1_11_1","first-page":"188","article-title":"Search engine bias and the demise of search engine utopianism","volume":"8","author":"Goldman Eric","year":"2005","unstructured":"Eric Goldman . Search engine bias and the demise of search engine utopianism . Yale Journal of Law & Technology , 8 : 188 , 2005 . Eric Goldman. Search engine bias and the demise of search engine utopianism. Yale Journal of Law & Technology, 8:188, 2005.","journal-title":"Yale Journal of Law & Technology"},{"key":"e_1_2_1_12_1","volume-title":"Data voids: Where missing data can easily be exploited","author":"Golebiewski Michael","year":"2018","unstructured":"Michael Golebiewski and danah boyd. Data voids: Where missing data can easily be exploited . Data & Society Research Institute , 2018 . Michael Golebiewski and danah boyd. Data voids: Where missing data can easily be exploited. Data & Society Research Institute, 2018."},{"key":"e_1_2_1_13_1","first-page":"1","article-title":"Technology-assisted review in e-discovery can be more effective and more efficient than exhaustive manual review","volume":"17","author":"Grossman Maura R","year":"2010","unstructured":"Maura R Grossman and Gordon V Cormack . Technology-assisted review in e-discovery can be more effective and more efficient than exhaustive manual review . Richmond Journal of Law and Technology , 17 : 1 , 2010 . Maura R Grossman and Gordon V Cormack. Technology-assisted review in e-discovery can be more effective and more efficient than exhaustive manual review. Richmond Journal of Law and Technology, 17:1, 2010.","journal-title":"Richmond Journal of Law and Technology"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998327"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287600"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1080\/01972240050133634"},{"key":"e_1_2_1_17_1","first-page":"50","volume-title":"Proceedings of the 7th AAAI Conference on Human Computation and Crowdsourcing (HCOMP)","author":"Karunakaran Sowmya","year":"2019","unstructured":"Sowmya Karunakaran and Rashmi Ramakrishan . Testing stylistic interventions to reduce emotional impact of content moderation workers . In Proceedings of the 7th AAAI Conference on Human Computation and Crowdsourcing (HCOMP) , pages 50 -- 58 , 2019 . Sowmya Karunakaran and Rashmi Ramakrishan. Testing stylistic interventions to reduce emotional impact of content moderation workers. In Proceedings of the 7th AAAI Conference on Human Computation and Crowdsourcing (HCOMP), pages 50--58, 2019."},{"key":"e_1_2_1_18_1","volume-title":"Right to be forgotten online could spread. New York Times","author":"Manjoo Farhad","year":"2015","unstructured":"Farhad Manjoo . Right to be forgotten online could spread. New York Times , 2015 . URL \\url{https:\/\/www.nytimes.com\/2015\/08\/06\/technology\/personaltech\/right-to-be-forgotten-online-is-poised-to-spread.html}. Farhad Manjoo. Right to be forgotten online could spread. New York Times, 2015. URL\\url{https:\/\/www.nytimes.com\/2015\/08\/06\/technology\/personaltech\/right-to-be-forgotten-online-is-poised-to-spread.html}."},{"key":"e_1_2_1_19_1","volume-title":"ECIR 2018, Grenoble, France, March 26--29, 2018, Proceedings","volume":"10772","author":"McDonald Graham","year":"2018","unstructured":"Graham McDonald , Craig Macdonald , and Iadh Ounis . Active learning strategies for technology assisted sensitivity review. In Advances in Information Retrieval - 40th European Conference on IR Research , ECIR 2018, Grenoble, France, March 26--29, 2018, Proceedings , volume 10772 of Lecture Notes in Computer Science, pages 439--453. Springer , 2018 . Graham McDonald, Craig Macdonald, and Iadh Ounis. Active learning strategies for technology assisted sensitivity review. In Advances in Information Retrieval - 40th European Conference on IR Research, ECIR 2018, Grenoble, France, March 26--29, 2018, Proceedings, volume 10772 of Lecture Notes in Computer Science, pages 439--453. Springer, 2018."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3041021.3054197"},{"key":"e_1_2_1_21_1","volume-title":"Explainable ai: Beware of inmates running the asylum or: How I learnt to stop worrying and love the social and behavioural sciences. arXiv preprint arXiv:1712.00547","author":"Miller Tim","year":"2017","unstructured":"Tim Miller , Piers Howe , and Liz Sonenberg . Explainable ai: Beware of inmates running the asylum or: How I learnt to stop worrying and love the social and behavioural sciences. arXiv preprint arXiv:1712.00547 , 2017 . Tim Miller, Piers Howe, and Liz Sonenberg. Explainable ai: Beware of inmates running the asylum or: How I learnt to stop worrying and love the social and behavioural sciences. arXiv preprint arXiv:1712.00547, 2017."},{"key":"e_1_2_1_22_1","volume-title":"Tenth International AAAI Conference on Web and Social Media","author":"Nilizadeh Shirin","year":"2016","unstructured":"Shirin Nilizadeh , Anne Groggel , Peter Lista , Srijita Das , Yong-Yeol Ahn , Apu Kapadia , and Fabio Rojas . Twitter's glass ceiling : The effect of perceived gender on online visibility . In Tenth International AAAI Conference on Web and Social Media , 2016 . Shirin Nilizadeh, Anne Groggel, Peter Lista, Srijita Das, Yong-Yeol Ahn, Apu Kapadia, and Fabio Rojas. Twitter's glass ceiling: The effect of perceived gender on online visibility. In Tenth International AAAI Conference on Web and Social Media, 2016."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-010-9093-9"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.3389\/fdata.2019.00013"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of FACTS-IR 2019, 2019","author":"Olteanu Alexandra","year":"1907","unstructured":"Alexandra Olteanu , Jean Garcia-Gathright , Maarten de Rijke , and Michael D . Ekstrand, editors . Proceedings of FACTS-IR 2019, 2019 b. CoRR abs\/ 1907 .05755. URL http:\/\/arxiv.org\/abs\/1907.05755. Alexandra Olteanu, Jean Garcia-Gathright, Maarten de Rijke, and Michael D. Ekstrand, editors. Proceedings of FACTS-IR 2019, 2019b. CoRR abs\/1907.05755. URL http:\/\/arxiv.org\/abs\/1907.05755."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331644"},{"key":"e_1_2_1_27_1","first-page":"628","article-title":"The mosaic theory, national security, and the freedom of information act","volume":"115","author":"Pozen David E.","year":"2005","unstructured":"David E. Pozen . The mosaic theory, national security, and the freedom of information act . The Yale Law Journal , 115 : 628 -- 679 , 2005 . Available at: https:\/\/scholarship.law.columbia.edu\/faculty_scholarship\/573. David E. Pozen. The mosaic theory, national security, and the freedom of information act. The Yale Law Journal, 115:628--679, 2005. Available at: https:\/\/scholarship.law.columbia.edu\/faculty_scholarship\/573.","journal-title":"The Yale Law Journal"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441955.2441981"},{"key":"e_1_2_1_29_1","first-page":"88","article-title":"The right to be forgotten","volume":"64","author":"Rosen Jeffrey","year":"2011","unstructured":"Jeffrey Rosen . The right to be forgotten . Stanford Law Review , 64 : 88 , 2011 . Jeffrey Rosen. The right to be forgotten. Stanford Law Review, 64:88, 2011.","journal-title":"Stanford Law Review"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979127"},{"key":"e_1_2_1_31_1","first-page":"4584","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Sharchilev Boris","year":"2018","unstructured":"Boris Sharchilev , Yury Ustinovsky , Pavel Serdyukov , and Maarten de Rijke . Finding influential training samples for gradient boosted decision trees . In Proceedings of the International Conference on Machine Learning , pages 4584 -- 4592 , 2018 . Boris Sharchilev, Yury Ustinovsky, Pavel Serdyukov, and Maarten de Rijke. Finding influential training samples for gradient boosted decision trees. In Proceedings of the International Conference on Machine Learning, pages 4584--4592, 2018."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.5555\/1355304"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186140"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1093\/idpl\/ipy002"},{"issue":"2133","key":"e_1_2_1_35_1","first-page":"20180083","article-title":"Algorithms that remember: model inversion attacks and data protection law. Philosophical Transactions of the Royal Society A: Mathematical","volume":"376","author":"Veale Michael","year":"2018","unstructured":"Michael Veale , Reuben Binns , and Lilian Edwards . Algorithms that remember: model inversion attacks and data protection law. Philosophical Transactions of the Royal Society A: Mathematical , Physical and Engineering Sciences , 376 ( 2133 ): 20180083 , 2018 b. Michael Veale, Reuben Binns, and Lilian Edwards. Algorithms that remember: model inversion attacks and data protection law. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 376(2133):20180083, 2018b.","journal-title":"Physical and Engineering Sciences"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2018.23036"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484053"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2917763"}],"container-title":["ACM SIGIR Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3458553.3458556","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3458553.3458556","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:17:17Z","timestamp":1750191437000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3458553.3458556"}},"subtitle":["fairness, accountability, confidentiality, transparency, and safety in information retrieval"],"short-title":[],"issued":{"date-parts":[[2019,12]]},"references-count":38,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["10.1145\/3458553.3458556"],"URL":"https:\/\/doi.org\/10.1145\/3458553.3458556","relation":{},"ISSN":["0163-5840"],"issn-type":[{"value":"0163-5840","type":"print"}],"subject":[],"published":{"date-parts":[[2019,12]]},"assertion":[{"value":"2021-03-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}