{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T20:29:21Z","timestamp":1792787361777,"version":"4.5.0"},"publisher-location":"New York, NY, USA","reference-count":84,"publisher":"ACM","license":[{"start":{"date-parts":[[2018,4,21]],"date-time":"2018-04-21T00:00:00Z","timestamp":1524268800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"EPSRC","award":["EP\/M507970\/1"],"award-info":[{"award-number":["EP\/M507970\/1"]}],"id":[{"id":"https:\/\/ror.org\/0439y7842","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2018,4,21]]},"DOI":"10.1145\/3173574.3174014","type":"proceedings-article","created":{"date-parts":[[2018,4,20]],"date-time":"2018-04-20T11:38:35Z","timestamp":1524224315000},"page":"1-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":357,"title":["Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making"],"prefix":"10.1145","author":[{"given":"Michael","family":"Veale","sequence":"first","affiliation":[{"name":"University College London, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Max","family":"Van Kleek","sequence":"additional","affiliation":[{"name":"University of Oxford, Oxford, Oxfordshire, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Reuben","family":"Binns","sequence":"additional","affiliation":[{"name":"University of Oxford, Oxford, Oxfordshire, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,4,21]]},"reference":[{"key":"e_1_3_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1080\/13658816.2016.1159684"},{"key":"e_1_3_2_2_2_1","volume-title":"The UK Administrative Data Research Network: Improving access for research and policy","author":"Taskforce Administrative Data","unstructured":"Administrative Data Taskforce. 2012. The UK Administrative Data Research Network: Improving access for research and policy. Economic and Social Research Council. http:\/\/www.esrc.ac.uk\/files\/ research\/administrative-data-taskforce-adt\/ improving-access-for-research-and-policy\/"},{"key":"e_1_3_2_2_3_1","unstructured":"AI Now. 2016. The AI Now Report: The Social and Economic Implications of Artificial Intelligence Technologies in the Near-Term. https:\/\/artificialintelligencenow.com\/"},{"key":"e_1_3_2_2_4_1","volume-title":"Machine bias. ProPublica","author":"Angwin Julia","year":"2016","unstructured":"Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica (2016). https:\/\/www.propublica.org\/article\/ machine-bias-risk-assessments-in-criminal-sentencing"},{"key":"e_1_3_2_2_5_1","first-page":"671","article-title":"Big Data's Disparate Impact","volume":"104","author":"Barocas Solon","year":"2016","unstructured":"Solon Barocas and Andrew D Selbst. 2016. Big Data's Disparate Impact. California Law Review 104 (2016), 671--732.","journal-title":"California Law Review"},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1177\/026839629601100305"},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9299.2006.00600.x"},{"key":"e_1_3_2_2_8_1","volume-title":"The emergence of risk-based regulation and the new public risk management in the United Kingdom. Public Law","author":"Black Julia","year":"2005","unstructured":"Julia Black. 2005. The emergence of risk-based regulation and the new public risk management in the United Kingdom. Public Law (2005), 512--549. Issue Autumn. https:\/\/perma.cc\/Z8AU-4VNN"},{"key":"e_1_3_2_2_9_1","first-page":"226","article-title":"Undoing the neutrality of Big Data","volume":"16","year":"2016","unstructured":"danah boyd. 2016. Undoing the neutrality of Big Data. Florida Law Review Forum 16 (2016), 226--232.","journal-title":"Florida Law Review Forum"},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1080\/14719037.2013.771699"},{"key":"e_1_3_2_2_11_1","volume-title":"Workshop at the crossroads: The interaction of HCI and systems issues in UbiComp","volume":"8","author":"Chalmers Matthew","year":"2003","unstructured":"Matthew Chalmers and Ian MacColl. 2003. Seamful and seamless design in ubiquitous computing. In Workshop at the crossroads: The interaction of HCI and systems issues in UbiComp, Vol. 8. https:\/\/perma.cc\/2A3D-NMJP"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1056808.1056893"},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1089\/big.2016.0047"},{"key":"e_1_3_2_2_14_1","first-page":"1147","article-title":"Regulating by Robot: Administrative Decision Making in the Machine-Learning Era","volume":"105","author":"Coglianese Cary","year":"2016","unstructured":"Cary Coglianese and David Lehr. 2016. Regulating by Robot: Administrative Decision Making in the Machine-Learning Era. Geo. LJ 105 (2016), 1147. https:\/\/ssrn.com\/abstract=2928293","journal-title":"Geo. LJ"},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1006\/ijhc.1994.1083"},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.5555\/1213575"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2090236.2090255"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1071-5819(03)00038-7"},{"key":"e_1_3_2_2_19_1","volume-title":"More accountability for big-data algorithms. Nature 537, 7621","year":"2016","unstructured":"Editor. 2016. More accountability for big-data algorithms. Nature 537, 7621 (2016), 449."},{"key":"e_1_3_2_2_20_1","first-page":"18","article-title":"Slave to the Algorithm? Why a 'Right to an Explanation' is Probably not the Remedy You are Looking For","volume":"16","author":"Edwards Lilian","year":"2017","unstructured":"Lilian Edwards and Michael Veale. 2017. Slave to the Algorithm? Why a 'Right to an Explanation' is Probably not the Remedy You are Looking For. Duke Law & Technology Review 16, 1 (2017), 18--84.","journal-title":"Duke Law & Technology Review"},{"key":"e_1_3_2_2_21_1","volume-title":"Runaway Feedback Loops in Predictive Policing. Presented as a talk at the 4th Workshop on Fairness, Accountability and Transparency in Machine Learning (FAT\/ML","author":"Ensign Danielle","year":"2017","unstructured":"Danielle Ensign, Sorelle A. Friedler, Scott Neville, Carlos Scheidegger, and Suresh Venkatasubramanian. 2017. Runaway Feedback Loops in Predictive Policing. Presented as a talk at the 4th Workshop on Fairness, Accountability and Transparency in Machine Learning (FAT\/ML 2017), Halifax, Canada (2017). https:\/\/arxiv.org\/abs\/1706.09847"},{"key":"e_1_3_2_2_22_1","volume-title":"SMART 2017\/0055","author":"European Commission","year":"2017","unstructured":"European Commission. 2017. Tender specifications: Study on Algorithmic Awareness Building, SMART 2017\/0055. https:\/\/etendering.ted.europa.eu\/cft\/ cft-document.html?docId=28267"},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783311"},{"key":"e_1_3_2_2_24_1","volume-title":"The importance of models in making complex systems comprehensible","author":"Fischer Gerhard","unstructured":"Gerhard Fischer. 1991. The importance of models in making complex systems comprehensible. In Mental Models and Human-Computer Interaction, MJ Tauber and D Ackermann (Eds.). Elsevier, Noord Holland."},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/0957-4174(94)E0032-P"},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/230538.230561"},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/32206.315753"},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2523813"},{"key":"e_1_3_2_2_29_1","volume-title":"Discrimination and privacy in the information society","author":"Gellert Rapha\u00ebl","unstructured":"Rapha\u00ebl Gellert, Katja de Vries, Paul de Hert, and Serge Gutwirth. 2013. A Comparative Analysis of Anti-Discrimination and Data Protection Legislations. In Discrimination and privacy in the information society, Bart Custers, Toon Calders, Bart Schermer, and Tal Zarsky (Eds.). Springer, Heidelberg."},{"key":"e_1_3_2_2_30_1","volume-title":"Data science ethical framework. HM Government","author":"Service Government Digital","unstructured":"Government Digital Service. 2015. Data science ethical framework. HM Government, London. https:\/\/www.gov.uk\/government\/publications\/ data-science-ethical-framework"},{"key":"e_1_3_2_2_31_1","volume-title":"Artificial intelligence: Opportunities and implications for the future of decision making. HM Government","author":"Science Government Office","unstructured":"Government Office for Science. 2016. Artificial intelligence: Opportunities and implications for the future of decision making. HM Government, London. https:\/\/www.gov.uk\/government\/publications\/ artificial-intelligence-an-overview-for-policy-makers"},{"key":"e_1_3_2_2_32_1","volume-title":"Discrimination and privacy in the information society","author":"Hajian Sara","unstructured":"Sara Hajian and Josep Domingo-Ferrer. 2012. Direct and indirect discrimination prevention methods. In Discrimination and privacy in the information society, Bart Custers, Toon Calders, Bart Schermer, and Tal Zarsky (Eds.). Springer, Berlin, Heidelberg, 241--254."},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1993060.1993065"},{"key":"e_1_3_2_2_34_1","volume-title":"Human factors contributions to knowledge elicitation. Human factors 50, 3","author":"Hoffman Robert R","year":"2008","unstructured":"Robert R Hoffman. 2008. Human factors contributions to knowledge elicitation. Human factors 50, 3 (2008), 481--488."},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1518\/001872098779480442"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9299.1991.tb00779.x"},{"key":"e_1_3_2_2_37_1","volume-title":"Human factors in air traffic control","author":"Hopkin V David","unstructured":"V David Hopkin. 1995. Human factors in air traffic control. CRC Press, London."},{"key":"e_1_3_2_2_38_1","volume-title":"The governance of problems: Puzzling, powering and participation","author":"Hoppe Robert","unstructured":"Robert Hoppe. 2011. The governance of problems: Puzzling, powering and participation. Policy Press."},{"key":"e_1_3_2_2_39_1","volume-title":"Robotics and artificial intelligence (HC 145)","author":"House of Common Science and Technology Committee","year":"2016","unstructured":"House of Common Science and Technology Committee. 2016. Robotics and artificial intelligence (HC 145). The House of Commons, London. http:\/\/www.publications.parliament.uk\/pa\/cm201617\/ cmselect\/cmsctech\/145\/145.pdf"},{"key":"e_1_3_2_2_40_1","volume-title":"The big data dilemma (HC 468)","author":"House of Commons Science and Technology Committee","year":"2015","unstructured":"House of Commons Science and Technology Committee. 2016. The big data dilemma (HC 468). House of Commons, London. http:\/\/www.publications.parliament. uk\/pa\/cm201516\/cmselect\/cmsctech\/468\/468.pdf"},{"key":"e_1_3_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2046684.2046692"},{"key":"e_1_3_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.5555\/1964882"},{"key":"e_1_3_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1177\/0095399707300703"},{"key":"e_1_3_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9299.2007.00640.x"},{"key":"e_1_3_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-011-0463-8"},{"key":"e_1_3_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2010.50"},{"key":"e_1_3_2_2_47_1","volume-title":"Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 330, 7494","author":"Kawamoto Kensaku","year":"2005","unstructured":"Kensaku Kawamoto, Caitlin A Houlihan, E Andrew Balas, and David F Lobach. 2005. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 330, 7494 (2005), 765."},{"key":"e_1_3_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/506443.506491"},{"key":"e_1_3_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2738210.2738216"},{"key":"e_1_3_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11077-016-9250-4"},{"key":"e_1_3_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753345"},{"key":"e_1_3_2_2_52_1","volume-title":"Street-level bureaucracy: Dilemmas of the individual in public services","author":"Lipsky Michael","unstructured":"Michael Lipsky. 2010. Street-level bureaucracy: Dilemmas of the individual in public services. Russell Sage Foundation, New York."},{"key":"e_1_3_2_2_53_1","volume-title":"The Mythos of Model Interpretability. In 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI","author":"Lipton Zachary C","year":"2016","unstructured":"Zachary C Lipton. 2016. The Mythos of Model Interpretability. In 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016). New York. https:\/\/arxiv.org\/abs\/1606.03490"},{"key":"e_1_3_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2016.11.008"},{"key":"e_1_3_2_2_55_1","unstructured":"Robin Moore (Ed.). 2015. A compendium of research and analysis on the Offender Assessment System. Ministry of Justice Analytical Series London. DOI: http:\/\/dx.doi.org\/https:\/\/perma.cc\/W2FT-NFWZ"},{"key":"e_1_3_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.5555\/2666036.2666037"},{"key":"e_1_3_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15327108ijap0801_3"},{"key":"e_1_3_2_2_58_1","volume-title":"Machines that learn in the wild: Machine learning capabilities, limitations and implications","unstructured":"Nesta. 2015. Machines that learn in the wild: Machine learning capabilities, limitations and implications. Nesta, London. https:\/\/perma.cc\/A6AM-GV6X"},{"key":"e_1_3_2_2_59_1","volume-title":"Kent slavery raids 'uncover 21 victims'. BBC News (7","author":"News BBC","year":"2016","unstructured":"BBC News. 2016. Kent slavery raids 'uncover 21 victims'. BBC News (7 Dec. 2016). https:\/\/perma.cc\/AM4S-RMHR"},{"key":"e_1_3_2_2_60_1","volume-title":"Some observations on mental models","author":"Norman Donald A","unstructured":"Donald A Norman. 1983. Some observations on mental models. In Mental Models, Dedre Gentner and Albert L Stevens (Eds.). Psychology Press, New York City, NY, 7--14."},{"key":"e_1_3_2_2_61_1","volume-title":"Interviewing elites. Handbook of Interview Research","author":"Odendahl Teresa","year":"2002","unstructured":"Teresa Odendahl and Aileen M Shaw. 2002. Interviewing elites. Handbook of Interview Research (2002), 299--316."},{"key":"e_1_3_2_2_62_1","unstructured":"Marion Oswald Jamie Grace Sheena Urwin and Geoffrey C. Barnes. forthcoming. Algorithmic Risk Assessment Policing Models: Lessons from the Durham Hart Model and 'Experimental' Proportionality. Information & Communications Technology Laws (forthcoming). https:\/\/ssrn.com\/abstract=3029345"},{"key":"e_1_3_2_2_63_1","volume-title":"Policy bureaucracy: Government with a cast of thousands","author":"Page Edward C","unstructured":"Edward C Page and Bill Jenkins. 2005. Policy bureaucracy: Government with a cast of thousands. Oxford University Press, Oxford."},{"key":"e_1_3_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/1401890.1401959"},{"key":"e_1_3_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.5555\/1462129"},{"key":"e_1_3_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_2_67_1","volume-title":"Proceedings of the 28th International Conference on Neural Information Processing Systems, Montr\u00e9al, Canada - December 07--12","author":"Sculley D","year":"2015","unstructured":"D Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-Francois Crespo, and Dan Dennison. 2015. Hidden Technical Debt in Machine Learning Systems. In Proceedings of the 28th International Conference on Neural Information Processing Systems, Montr\u00e9al, Canada - December 07--12, 2015. MIT Press, Cambridge, MA, 2503--2511. https:\/\/perma.cc\/G6VN-9B86"},{"key":"e_1_3_2_2_68_1","volume-title":"Knowing algorithms. Media in Transition 8","author":"Seaver Nick","year":"2013","unstructured":"Nick Seaver. 2013. Knowing algorithms. Media in Transition 8 (2013). https:\/\/perma.cc\/8USJ-VTWS"},{"key":"e_1_3_2_2_69_1","volume-title":"On reverse engineering: Looking for the cultural work of engineers {blog. Medium","author":"Seaver Nick","year":"2014","unstructured":"Nick Seaver. 2014. On reverse engineering: Looking for the cultural work of engineers {blog. Medium (2014). https:\/\/medium.com\/anthropology-and-algorithms\/ on-reverse-engineering-d9f5bae87812"},{"key":"e_1_3_2_2_70_1","unstructured":"Andrew Selbst. forthcoming. Disparate Impact in Big Data Policing. Georgia Law Review (forthcoming)."},{"key":"e_1_3_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.1006\/ijhc.1999.0252"},{"key":"e_1_3_2_2_72_1","volume-title":"Machine learning: The power and promise of computers that learn by example","author":"The Royal Society","unstructured":"The Royal Society. 2017. Machine learning: The power and promise of computers that learn by example. The Royal Society, London. https:\/\/royalsociety.org\/~\/ media\/policy\/projects\/machine-learning\/publications\/ machine-learning-report.pdf"},{"key":"e_1_3_2_2_73_1","volume-title":"Data management and use: Governance in the 21st Century","author":"The Royal Society and the British Academy. 2017.","unstructured":"The Royal Society and the British Academy. 2017. Data management and use: Governance in the 21st Century. The Royal Society and the British Academy, London. https:\/\/royalsociety.org\/~\/media\/policy\/projects\/ data-governance\/data-management-governance.pdf"},{"key":"e_1_3_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.0272-4332.2004.00552.x"},{"key":"e_1_3_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1109\/72.728352"},{"key":"e_1_3_2_2_76_1","doi-asserted-by":"publisher","DOI":"10.1177\/0759106315615504"},{"key":"e_1_3_2_2_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240630"},{"key":"e_1_3_2_2_78_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-015-5528-6"},{"key":"e_1_3_2_2_79_1","volume-title":"Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data","author":"Veale Michael","year":"2017","unstructured":"Michael Veale and Reuben Binns. 2017. Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data. Big Data & Society 4, 2 (2017)."},{"key":"e_1_3_2_2_80_1","unstructured":"Wetenschappelijke Raad voor het Regeringsbeleid. 2016. Big Data in een vrije en veilige samenleving (WRR-Rapport 95). WRR Den Haag. http:\/\/www.wrr.nl\/publicaties\/publicatie\/article\/ big-data-in-een-vrije-en-veilige-samenleving\/"},{"key":"e_1_3_2_2_81_1","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(92)90087-E"},{"key":"e_1_3_2_2_82_1","first-page":"121","article-title":"Do Artifacts Have Politics","volume":"109","author":"Winner Langdon","year":"1980","unstructured":"Langdon Winner. 1980. Do Artifacts Have Politics? D\u00e6delus 109, 1 (1980), 121--136. http:\/\/www.jstor.org\/stable\/20024652","journal-title":"D\u00e6delus"},{"key":"e_1_3_2_2_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858373"},{"key":"e_1_3_2_2_84_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702239"}],"event":{"name":"CHI '18: CHI Conference on Human Factors in Computing Systems","location":"Montreal QC Canada","acronym":"CHI '18","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3173574.3174014","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3173574.3174014","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,21]],"date-time":"2026-09-21T14:16:57Z","timestamp":1790000217000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3173574.3174014"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,4,21]]},"references-count":84,"alternative-id":["10.1145\/3173574.3174014","10.1145\/3173574"],"URL":"https:\/\/doi.org\/10.1145\/3173574.3174014","relation":{"has-preprint":[{"id-type":"doi","id":"10.31235\/osf.io\/8kvf4","asserted-by":"object"}]},"subject":[],"published":{"date-parts":[[2018,4,21]]},"assertion":[{"value":"2018-04-21","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}