{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T00:53:05Z","timestamp":1776300785709,"version":"3.50.1"},"reference-count":108,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2022,1,1]],"date-time":"2022-01-01T00:00:00Z","timestamp":1640995200000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Big Data &amp; Society"],"published-print":{"date-parts":[[2022,1]]},"abstract":"<jats:p> The algorithmic accountability literature to date has primarily focused on procedural tools to govern automated decision-making systems. That prescriptive literature elides a fundamentally empirical question: whether and under what circumstances, if any, is the use of algorithmic systems to make public policy decisions perceived as legitimate? The present study begins to answer this question. Using factorial vignette survey methodology, we explore the relative importance of the type of decision, the procedural governance, the input data used, and outcome errors on perceptions of the legitimacy of algorithmic public policy decisions as compared to similar human decisions. Among other findings, we find that the type of decision\u2014low importance versus high importance\u2014impacts the perceived legitimacy of automated decisions. We find that human governance of algorithmic systems (aka human-in-the-loop) increases perceptions of the legitimacy of algorithmic decision-making systems, even when those decisions are likely to result in significant errors. Notably, we also find the penalty to perceived legitimacy is greater when human decision-makers make mistakes than when algorithmic systems make the same errors. The positive impact on perceived legitimacy from governance\u2014such as human-in-the-loop\u2014is greatest for highly pivotal decisions such as parole, policing, and healthcare. After discussing the study\u2019s limitations, we outline avenues for future research. <\/jats:p>","DOI":"10.1177\/20539517221100449","type":"journal-article","created":{"date-parts":[[2022,6,29]],"date-time":"2022-06-29T05:15:13Z","timestamp":1656479713000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":44,"title":["Governing algorithmic decisions: The role of decision importance and governance on perceived legitimacy of algorithmic decisions"],"prefix":"10.1177","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9264-9231","authenticated-orcid":false,"given":"Ari","family":"Waldman","sequence":"first","affiliation":[{"name":"Northeastern University, School of Law and Khoury College of Computer Sciences, Boston, MA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0638-0169","authenticated-orcid":false,"given":"Kirsten","family":"Martin","sequence":"additional","affiliation":[{"name":"University of Notre Dame, Mendoza College of Business, Notre Dame, IN, USA"}]}],"member":"179","published-online":{"date-parts":[[2022,6,28]]},"reference":[{"key":"bibr1-20539517221100449","first-page":"1671","volume":"41","author":"Ajunwa I","year":"2020","journal-title":"Cardozo Law Review"},{"key":"bibr2-20539517221100449","doi-asserted-by":"publisher","DOI":"10.17265\/2159-5313\/2016.09.003"},{"key":"bibr3-20539517221100449","first-page":"137","volume-title":"Improving Survey Methods: Lessons from Recent Research","author":"Auspurg K","year":"2014"},{"key":"bibr200-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1086\/705385"},{"key":"bibr201-20539517221100449","first-page":"2","volume":"1","author":"Barocas S","year":"2017","journal-title":"Nips tutorial"},{"key":"bibr202-20539517221100449","volume":"104","author":"Barocas S","year":"2016","journal-title":"California Law Review"},{"key":"bibr4-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1111\/j.1540-5907.2012.00616.x"},{"key":"bibr5-20539517221100449","volume-title":"Race After Technology","author":"Benjamin R","year":"2019"},{"key":"bibr6-20539517221100449","first-page":"1277","volume":"98","author":"Berman E","year":"2018","journal-title":"Boston University Law Review"},{"key":"bibr204-20539517221100449","doi-asserted-by":"publisher","DOI":"10.5465\/amr.2013.0318"},{"key":"bibr7-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1111\/1467-9760.00061"},{"key":"bibr8-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1515\/9781503602304"},{"key":"bibr9-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1108\/CCIJ-12-2015-0085"},{"key":"bibr10-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1177\/2053951715622512"},{"key":"bibr11-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1126\/science.aal4230"},{"key":"bibr12-20539517221100449","first-page":"399","volume":"51","author":"Calo R","year":"2017","journal-title":"University of California, Davis Law Review"},{"key":"bibr13-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1111\/ajps.12150"},{"issue":"6","key":"bibr14-20539517221100449","first-page":"1249","volume":"85","author":"Citron D","year":"2007","journal-title":"Washington University Law Review"},{"issue":"1","key":"bibr15-20539517221100449","first-page":"1","volume":"89","author":"Citron D","year":"2014","journal-title":"Washington Law Review"},{"key":"bibr225-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1093\/oso\/9780190246693.001.0001"},{"key":"bibr16-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1007\/s00146-021-01184-2"},{"key":"bibr17-20539517221100449","doi-asserted-by":"crossref","unstructured":"Coppock A (2018) Generalizing from Survey Experiments Conducted on Mechanical Turk: A replication approach. Political Science Research and Methods 7(3): 613\u2013628","DOI":"10.1017\/psrm.2018.10"},{"key":"bibr18-20539517221100449","volume-title":"Introduction to Algorithms","author":"Cormen T","year":"2009"},{"issue":"1","key":"bibr19-20539517221100449","first-page":"93","volume":"55","author":"Crawford K","year":"2014","journal-title":"Boston College Law Review"},{"key":"bibr20-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbusres.2015.05.001"},{"key":"bibr21-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1177\/2053951717726554"},{"key":"bibr22-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1007\/s00146-020-00960-w"},{"key":"bibr23-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1007\/s10676-019-09509-y"},{"key":"bibr24-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1080\/01449290600874956"},{"key":"bibr26-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.aao5580"},{"key":"bibr203-20539517221100449","volume-title":"Freedom\u2019s Law","author":"Dworkin R","year":"1996"},{"key":"bibr27-20539517221100449","volume-title":"A Systems Analysis of Political Life","author":"Easton D","year":"1965"},{"key":"bibr28-20539517221100449","volume-title":"Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor","author":"Eubanks V","year":"2018"},{"key":"bibr29-20539517221100449","doi-asserted-by":"publisher","DOI":"10.17265\/2159-5313\/2016.09.003"},{"key":"bibr30-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1145\/3461702.3462470"},{"key":"bibr31-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1515\/9781400830602"},{"key":"bibr32-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1111\/1540-5907.00025"},{"key":"bibr205-20539517221100449","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3859358"},{"key":"bibr34-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1111\/j.1475-6765.2006.00299.x"},{"key":"bibr35-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1111\/j.1540-5893.2008.00353.x"},{"key":"bibr36-20539517221100449","unstructured":"Hao K (2020) The UK Exam Debacle Reminds Us that Algorithms Can\u2019t Fix Broken Systems. MIT Technology Review, https:\/\/www.technologyreview.com\/2020\/08\/20\/1007502\/uk-exam-algorithm-cant-fix-broken-system\/"},{"key":"bibr38-20539517221100449","first-page":"527","volume":"15","author":"Henderson S","year":"2018","journal-title":"Ohio State Journal of Criminal Law"},{"key":"bibr39-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1177\/0018720814547570"},{"key":"bibr40-20539517221100449","unstructured":"Houston Federation of Teachers, Local 2415 v. Houston Independent School District, 251 F. Supp. 3d 1168 (S.D. Tex. 2017)."},{"issue":"5","key":"bibr41-20539517221100449","first-page":"1735","volume":"67","author":"Hu M","year":"2016","journal-title":"Florida Law Review"},{"key":"bibr42-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1007\/s00146-020-01130-8"},{"key":"bibr43-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445923"},{"key":"bibr44-20539517221100449","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.968350"},{"key":"bibr45-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1177\/0049124105283121"},{"key":"bibr46-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1177\/0306312717699716"},{"key":"bibr47-20539517221100449","unstructured":"Joseph G, Lipp K (2018) IBM Used NYPD Surveillance Footage to Develop Technology That Lets Police Search by Skin Color. The Intercept, https:\/\/theintercept.com\/2018\/09\/06\/nypd-surveillance-camera-skin-tone-search\/"},{"key":"bibr48-20539517221100449","doi-asserted-by":"crossref","unstructured":"Jung J, Concannon C, Shroff R, et al. (2017) Simple Rules for Complex Decisions. Working Paper, Stanford University, https:\/\/arxiv.org\/pdf\/1702.04690.pdf. Accessed December 4, 2019.","DOI":"10.2139\/ssrn.2919024"},{"key":"bibr49-20539517221100449","unstructured":"K.W. v. Armstrong, 180 F. Supp. 3d 703 (D. Idaho 2016)."},{"key":"bibr50-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1111\/j.1475-6765.2008.00773.x"},{"issue":"6","key":"bibr51-20539517221100449","first-page":"1529","volume":"92","author":"Kaminski M","journal-title":"Southern California Law Review"},{"key":"bibr52-20539517221100449","first-page":"189","volume":"34","author":"Kaminski M","journal-title":"Berkeley Technology Law Journal"},{"key":"bibr206-20539517221100449","unstructured":"Kang R, Brown S, Dabbish L, et al. (2014) Privacy Attitudes of Mechanical Turk Workers and the US Public. In: SOUPS, 2014, pp. 37\u201349."},{"key":"bibr53-20539517221100449","first-page":"54","volume":"66","author":"Katyal S","year":"2019","journal-title":"University of California, Los Angeles Law Review"},{"key":"bibr54-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1177\/20539517211069632"},{"key":"bibr55-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1007\/s10676-010-9233-7"},{"key":"bibr56-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1177\/20539517211049670"},{"key":"bibr57-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1177\/2053951718756684"},{"key":"bibr58-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1007\/s10551-019-04204-w"},{"key":"bibr59-20539517221100449","doi-asserted-by":"publisher","DOI":"10.2307\/1951731"},{"key":"bibr60-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1007\/s10676-020-09564-w"},{"key":"bibr62-20539517221100449","doi-asserted-by":"crossref","unstructured":"L\u00fcnich M, Kieslich K (2021) Using automated decision-making (ADM) to allocate COVID-19 vaccinations? Exploring the roles of trust and social group preference on the legitimacy of ADM vs. Human decision-making. arXiv preprint arXiv:2107.08946: 1\u201317.","DOI":"10.1007\/s00146-022-01412-3"},{"key":"bibr211-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1086\/208919"},{"key":"bibr63-20539517221100449","first-page":"53","volume":"95","author":"Madden M","year":"2017","journal-title":"Washington University Law Review"},{"key":"bibr64-20539517221100449","first-page":"1","volume":"160","author":"Martin K","year":"2018","journal-title":"Journal of Business Ethics"},{"key":"bibr65-20539517221100449","doi-asserted-by":"publisher","DOI":"10.17705\/2msqe.00012"},{"issue":"1","key":"bibr66-20539517221100449","first-page":"111","volume":"31","author":"Martin K","year":"2017","journal-title":"Harvard Journal of Law and Technology"},{"issue":"1","key":"bibr67-20539517221100449","first-page":"251","volume":"35","author":"Martin K","year":"2020","journal-title":"Berkeley Technology Law Journal"},{"key":"bibr68-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445935"},{"key":"bibr69-20539517221100449","doi-asserted-by":"publisher","DOI":"10.18574\/nyu\/9781479833641.001.0001"},{"key":"bibr70-20539517221100449","volume-title":"Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy","author":"O\u2019Neil C","year":"2016"},{"key":"bibr71-20539517221100449","first-page":"1701","volume":"57","author":"Ohm P","year":"2010","journal-title":"UCLA Law Review"},{"key":"bibr72-20539517221100449","volume-title":"Black Box Society: The Secret Algorithms That Control Money and Information","author":"Pasquale F","year":"2014"},{"issue":"7","key":"bibr73-20539517221100449","first-page":"1917","volume":"119","author":"Pasquale F","year":"2019","journal-title":"Columbia Law Review"},{"key":"bibr207-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1017\/S1755773912000173"},{"key":"bibr74-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1177\/0007650316647950"},{"key":"bibr75-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1007\/s10551-015-2839-2"},{"key":"bibr76-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1007\/s10676-017-9430-8"},{"key":"bibr209-20539517221100449","unstructured":"Redmiles EM, Kross S, Pradhan A, et al. (2017) How well do my results generalize? Comparing security and privacy survey results from MTurk and web panels to the US."},{"key":"bibr77-20539517221100449","unstructured":"Reisman D, Schultz J, Crawford K, et al. (2018) \u201cAlgorithmic Impact Assessments: A Practical Framework for Public Agency Accountability.\u201d AI Now Institute. https:\/\/ainowinstitute.org\/aiareport2018.pdf"},{"key":"bibr78-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1915006117"},{"key":"bibr79-20539517221100449","first-page":"1085","volume":"87","author":"Selbst A","year":"2018","journal-title":"Fordham Law Review"},{"key":"bibr210-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1093\/jcr\/ucx053"},{"key":"bibr80-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1007\/s10676-018-9489-x"},{"key":"bibr81-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1037\/0021-9010.82.3.434"},{"key":"bibr82-20539517221100449","unstructured":"Sonnad N (2018) US Border Agents Hacked Their \u2018Risk Assessment\u2019 System to Recommend Detention 100% of the Time. Quartz, https:\/\/qz.com\/1314749\/us-border-agents-hacked-their-risk-assessment-system-to-recommend-immigrant-detention-every-time\/"},{"key":"bibr83-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1017\/dap.2020.19"},{"key":"bibr84-20539517221100449","doi-asserted-by":"publisher","DOI":"10.2307\/258788"},{"key":"bibr86-20539517221100449","doi-asserted-by":"crossref","unstructured":"S\u00fchr T, Hilgard S, Lakkaraju H (2021) Does fair ranking improve minority outcomes? understanding the interplay of human and algorithmic biases in online hiring. In Proceedings of the 2021 AAAI\/ACM Conference on AI, Ethics, and Society: 989\u2013999.","DOI":"10.1145\/3461702.3462602"},{"key":"bibr87-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1111\/1540-5893.3703002"},{"key":"bibr88-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1287\/mksc.2014.0874"},{"key":"bibr89-20539517221100449","first-page":"203","volume":"13","author":"Tufekci Z","year":"2015","journal-title":"Colorado Journal of Telecommunications and High Technology"},{"key":"bibr91-20539517221100449","doi-asserted-by":"publisher","DOI":"10.2307\/3053998"},{"key":"bibr92-20539517221100449","volume-title":"Why People Obey the Law","author":"Tyler T"},{"key":"bibr93-20539517221100449","volume-title":"Trust in the Law: Encouraging Public Cooperation with the Police and Courts","author":"Tyler T","year":"2002"},{"key":"bibr94-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1177\/2053951717743530"},{"key":"bibr95-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1016\/j.ssresearch.2009.03.004"},{"key":"bibr96-20539517221100449","volume-title":"The Theory of Social and Economic Organizations","author":"Weber M"},{"key":"bibr97-20539517221100449","first-page":"1343","volume":"70","author":"Wexler R","year":"2018","journal-title":"Stanford Law Review"},{"key":"bibr98-20539517221100449","unstructured":"Whittaker M, et al. (2018) AI Now Report 2018. AI Now Institute. https:\/\/ainowinstitute.org\/AI_Now_2018_Report.pdf"},{"key":"bibr99-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-30487-3_17"},{"key":"bibr100-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1093\/ia\/iiaa172"},{"key":"bibr101-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1037\/0021-9010.90.3.553"},{"key":"bibr102-20539517221100449","doi-asserted-by":"publisher","DOI":"10.1177\/2053951714559253"}],"container-title":["Big Data &amp; Society"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/20539517221100449","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/20539517221100449","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/20539517221100449","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,1]],"date-time":"2025-03-01T09:57:36Z","timestamp":1740823056000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/20539517221100449"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1]]},"references-count":108,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1]]}},"alternative-id":["10.1177\/20539517221100449"],"URL":"https:\/\/doi.org\/10.1177\/20539517221100449","relation":{},"ISSN":["2053-9517","2053-9517"],"issn-type":[{"value":"2053-9517","type":"print"},{"value":"2053-9517","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1]]},"article-number":"20539517221100449"}}