{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T06:15:43Z","timestamp":1787897743278,"version":"build-2784847793"},"reference-count":76,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2022,10,18]],"date-time":"2022-10-18T00:00:00Z","timestamp":1666051200000},"content-version":"vor","delay-in-days":370,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["DGE1745303"],"award-info":[{"award-number":["DGE1745303"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,10,13]]},"abstract":"<jats:p>Governments are increasingly turning to algorithmic risk assessments when making important decisions, such as whether to release criminal defendants before trial. Policymakers assert that providing public servants with algorithmic advice will improve human risk predictions and thereby lead to better (e.g., fairer) decisions. Yet because many policy decisions require balancing risk-reduction with competing goals, improving the accuracy of predictions may not necessarily improve the quality of decisions. If risk assessments make people more attentive to reducing risk at the expense of other values, these algorithms would diminish the implementation of public policy even as they lead to more accurate predictions. Through an experiment with 2,140 lay participants simulating two high-stakes government contexts, we provide the first direct evidence that risk assessments can systematically alter how people factor risk into their decisions. These shifts counteracted the potential benefits of improved prediction accuracy. In the pretrial setting of our experiment, the risk assessment made participants more sensitive to increases in perceived risk; this shift increased the racial disparity in pretrial detention by 1.9%. In the government loans setting of our experiment, the risk assessment made participants more risk-averse; this shift reduced government aid by 8.3%. These results demonstrate the potential limits and harms of attempts to improve public policy by incorporating predictive algorithms into multifaceted policy decisions. If these observed behaviors occur in practice, presenting risk assessments to public servants would generate unexpected and unjust shifts in public policy without being subject to democratic deliberation or oversight.<\/jats:p>","DOI":"10.1145\/3479562","type":"journal-article","created":{"date-parts":[[2021,10,19]],"date-time":"2021-10-19T02:39:17Z","timestamp":1634611157000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":63,"title":["Algorithmic Risk Assessments Can Alter Human Decision-Making Processes in High-Stakes Government Contexts"],"prefix":"10.1145","volume":"5","author":[{"given":"Ben","family":"Green","sequence":"first","affiliation":[{"name":"University of Michigan, Ann Arbor, MI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yiling","family":"Chen","sequence":"additional","affiliation":[{"name":"Harvard University, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,10,18]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"If You Give a Judge a Risk Score: Evidence from Kentucky Bail Decisions","author":"Albright Alex","year":"2019","unstructured":"Alex Albright. 2019. If You Give a Judge a Risk Score: Evidence from Kentucky Bail Decisions. The John M. Olin Center for Law, Economics, and Business Fellows' Discussion Paper Series, Vol. 85 (2019). https:\/\/thelittledataset.com\/about_files\/albright_judge_score.pdf"},{"key":"e_1_2_1_2_1","volume-title":"ABA Standards for Criminal Justice: Pretrial Release","author":"American Bar Association","unstructured":"American Bar Association. 2007. ABA Standards for Criminal Justice: Pretrial Release 3rd ed.). https:\/\/www.americanbar.org\/groups\/criminal_justice\/publications\/criminal_justice_section_archive\/crimjust_standards_pretrialrelease_blk\/","edition":"3"},{"key":"e_1_2_1_3_1","volume-title":"What is Human-in-the-Loop Machine Learning?","year":"2019","unstructured":"Appen. 2019. What is Human-in-the-Loop Machine Learning? (2019). https:\/\/appen.com\/blog\/human-in-the-loop\/"},{"key":"e_1_2_1_4_1","unstructured":"Arnold Ventures. 2019. Public Safety Assessment FAQs (\u201cPSA 101\u201d). (2019). https:\/\/craftmediabucket.s3.amazonaws.com\/uploads\/Public-Safety-Assessment-101_190319_140124.pdf"},{"key":"e_1_2_1_5_1","unstructured":"James Austin. 2014. Evaluation of Broward County Jail Population: Current Trends and Recommended Options. (2014). https:\/\/www.clearinghouse.net\/chDocs\/public\/JC-FL-0008-0018.pdf"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1093\/socpro\/spaa004"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.32614\/RJ-2018-017"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v076.i01"},{"key":"e_1_2_1_9_1","unstructured":"Tianqi Chen Tong He Michael Benesty Vadim Khotilovich Yuan Tang Hyunsu Cho Kailong Chen Rory Mitchell Ignacio Cano Tianyi Zhou Mu Li Junyuan Xie Min Lin Yifeng Geng and Yutian Li. 2020. xgboost: Extreme Gradient Boosting. (2020). https:\/\/CRAN.R-project.org\/package=xgboost"},{"key":"e_1_2_1_10_1","volume-title":"Metrics at Work: Journalism and the Contested Meaning of Algorithms","author":"Christin Ang\u00e8le","unstructured":"Ang\u00e8le Christin. 2020. Metrics at Work: Journalism and the Contested Meaning of Algorithms .Princeton University Press."},{"key":"e_1_2_1_11_1","first-page":"1249","article-title":"Technological Due Process","volume":"85","author":"Citron Danielle Keats","year":"2008","unstructured":"Danielle Keats Citron. 2008. Technological Due Process. Washington University Law Review, Vol. 85, 6 (2008), 1249--1313. https:\/\/openscholarship.wustl.edu\/law_lawreview\/vol85\/iss6\/2\/","journal-title":"Washington University Law Review"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1257\/aer.20131314"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1017\/psrm.2018.10"},{"key":"e_1_2_1_14_1","unstructured":"Bo Cowgill. 2018. The Impact of Algorithms on Judicial Discretion: Evidence from Regression Discontinuities. (2018). http:\/\/www.columbia.edu\/ bc2656\/papers\/RecidAlgo.pdf"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376638"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1111\/1745--9133.12481"},{"key":"e_1_2_1_17_1","volume-title":"Singh","author":"Desmarais Sarah L.","year":"2013","unstructured":"Sarah L. Desmarais and Jay P. Singh. 2013. Risk Assessment Instruments Validated and Implemented in Correctional Settings in the United States. Council of State Governments Justice Center (2013). https:\/\/csgjusticecenter.org\/wp-content\/uploads\/2020\/02\/Risk-Assessment-Instruments-Validated-and-Implemented-in-Correctional-Settings-in-the-United-States.pdf"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1037\/xge0000033"},{"key":"e_1_2_1_19_1","first-page":"151","article-title":"Priming Risk: The Accessibility of Uncertainty in Public Policy Decision Making","volume":"34","author":"Eckles David L.","year":"2011","unstructured":"David L. Eckles and Brian F. Schaffner. 2011. Priming Risk: The Accessibility of Uncertainty in Public Policy Decision Making. Journal of Insurance Issues, Vol. 34, 2 (2011), 151--171. http:\/\/www.jstor.org\/stable\/41946320","journal-title":"Journal of Insurance Issues"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1002\/bdm.416"},{"key":"e_1_2_1_21_1","volume-title":"Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor","author":"Eubanks Virginia","year":"2018","unstructured":"Virginia Eubanks. 2018. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor .St. Martin's Press."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1013203451"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1093\/rof\/rfm034"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1099-0771(199903)12:1"},{"key":"e_1_2_1_25_1","volume-title":"5th Workshop on Fairness, Accountability, and Transparency in Machine Learning. https:\/\/www.fatml.org\/media\/documents\/fair_risk_assessments_criminal_justice.pdf","author":"Green Ben","year":"2018","unstructured":"Ben Green. 2018. `Fair' Risk Assessments: A Precarious Approach for Criminal Justice Reform. In 5th Workshop on Fairness, Accountability, and Transparency in Machine Learning. https:\/\/www.fatml.org\/media\/documents\/fair_risk_assessments_criminal_justice.pdf"},{"key":"e_1_2_1_26_1","volume-title":"The Smart Enough City: Putting Technology in Its Place to Reclaim Our Urban Future","author":"Green Ben","unstructured":"Ben Green. 2019. The Smart Enough City: Putting Technology in Its Place to Reclaim Our Urban Future .MIT Press."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372869"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287563"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359152"},{"key":"e_1_2_1_30_1","volume-title":"The False Comfort of Human Oversight as an Antidote to A.I. Harm. Slate","author":"Green Ben","year":"2021","unstructured":"Ben Green and Amba Kak. 2021. The False Comfort of Human Oversight as an Antidote to A.I. Harm. Slate (2021). https:\/\/slate.com\/technology\/2021\/06\/human-oversight-artificial-intelligence-laws.html"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372840"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359280"},{"key":"e_1_2_1_33_1","first-page":"777","article-title":"Inside the Judicial Mind","volume":"86","author":"Guthrie Chris","year":"2001","unstructured":"Chris Guthrie, Jeffrey J. Rachlinski, and Andrew J. Wistrich. 2001. Inside the Judicial Mind. Cornell Law Review, Vol. 86, 4 (2001), 777--830. https:\/\/scholarship.law.cornell.edu\/facpub\/814","journal-title":"Cornell Law Review"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1540--6261.2005.00737.x"},{"key":"e_1_2_1_35_1","volume-title":"115th Congress","author":"Harris Kamala","year":"2017","unstructured":"Kamala Harris and Rand Paul. 2017. Pretrial Integrity and Safety Act of 2017. 115th Congress (2017). https:\/\/www.congress.gov\/bill\/115th-congress\/senate-bill\/1593"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","unstructured":"Kenneth Holstein Bruce M. McLaren and Vincent Aleven. 2019. Designing for Complementarity: Teacher and Student Needs for Orchestration Support in AI-Enhanced Classrooms. In Artificial Intelligence in Education. 157--171. https:\/\/doi.org\/10.1007\/978--3-030--23204--7_14","DOI":"10.1007\/978--3-030--23204--7_14"},{"key":"e_1_2_1_37_1","unstructured":"Indiana Supreme Court. 2010. Malenchik v. State .928 N.E.2d 564."},{"key":"e_1_2_1_38_1","volume-title":"HB 463 -- Statement from the Sponsors. Criminal Law Reform: The First Year of HB 463","author":"Jensen Tom","year":"2012","unstructured":"Tom Jensen and John Tilley. 2012. HB 463 -- Statement from the Sponsors. Criminal Law Reform: The First Year of HB 463 (2012). https:\/\/cdn.ymaws.com\/www.kybar.org\/resource\/resmgr\/2012_Convention_Files\/ac2012_2.pdf"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.5555\/2343576.2343643"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-020-0232--8"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1177\/0003122419884917"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1093\/qje\/qjx032"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1257\/aer.p20151023"},{"key":"e_1_2_1_44_1","first-page":"1725","article-title":"Danger Ahead: Risk Assessment and the Future of Bail Reform","volume":"93","author":"Koepke John Logan","year":"2018","unstructured":"John Logan Koepke and David G. Robinson. 2018. Danger Ahead: Risk Assessment and the Future of Bail Reform. Washington Law Review, Vol. 93 (2018), 1725--1807. https:\/\/digitalcommons.law.uw.edu\/wlr\/vol93\/iss4\/4\/","journal-title":"Washington Law Review"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2470684"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287590"},{"key":"e_1_2_1_47_1","volume-title":"How We Analyzed the COMPAS Recidivism Algorithm. ProPublica","author":"Larson Jeff","year":"2016","unstructured":"Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. 2016. How We Analyzed the COMPAS Recidivism Algorithm. ProPublica (2016). https:\/\/www.propublica.org\/article\/how-we-analyzed-the-compas-recidivism-algorithm"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1002\/bdm.3960080302"},{"key":"e_1_2_1_49_1","volume-title":"Street-Level Bureaucracy: Dilemmas of the Individual in Public Services 30th Anniversary Expanded ed.)","author":"Lipsky Michael","unstructured":"Michael Lipsky. 2010. Street-Level Bureaucracy: Dilemmas of the Individual in Public Services 30th Anniversary Expanded ed.). Russell Sage Foundation."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.obhdp.2018.12.005"},{"key":"e_1_2_1_51_1","volume-title":"Hoffman","author":"Mahoney Barry","year":"2001","unstructured":"Barry Mahoney, Bruce D. Beaudin, John A. Carver III, Daniel B. Ryan, and Richard B. Hoffman. 2001. Pretrial Services Programs: Responsibilities and Potential. National Institute of Justice: Issues and Practices in Criminal Justice (2001). https:\/\/www.ncjrs.gov\/pdffiles1\/nij\/181939.pdf"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2019.02767"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.21105\/joss.01541"},{"key":"e_1_2_1_54_1","unstructured":"myFICO. 2016. Understanding FICO Scores. (2016). https:\/\/www.myfico.com\/Downloads\/Files\/myFICO_UYFS_Booklet.pdf"},{"key":"e_1_2_1_55_1","unstructured":"New Jersey Courts. 2017. One Year Criminal Justice Reform Report to the Governor and the Legislature. (2017). https:\/\/www.njcourts.gov\/courts\/assets\/criminal\/2017cjrannual.pdf"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2788629"},{"key":"e_1_2_1_57_1","first-page":"1195","article-title":"Does Unconscious Racial Bias Affect Trial Judges","volume":"84","author":"Rachlinski Jeffrey J.","year":"2008","unstructured":"Jeffrey J. Rachlinski, Sheri Lynn Johnson, Andrew J. Wistrich, and Chris Guthrie. 2008. Does Unconscious Racial Bias Affect Trial Judges. Notre Dame Law Review, Vol. 84 (2008), 1195--1246. https:\/\/scholarship.law.nd.edu\/ndlr\/vol84\/iss3\/4\/","journal-title":"Notre Dame Law Review"},{"key":"e_1_2_1_58_1","first-page":"521","article-title":"Gains, Losses, and Judges: Framing and the Judiciary","volume":"94","author":"Rachlinski Jeffrey J.","year":"2018","unstructured":"Jeffrey J. Rachlinski and Andrew J. Wistrich. 2018. Gains, Losses, and Judges: Framing and the Judiciary. Notre Dame Law Review, Vol. 94, 2 (2018), 521--582. https:\/\/scholarship.law.nd.edu\/ndlr\/vol94\/iss2\/2\/","journal-title":"Notre Dame Law Review"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10979-010--9218--4"},{"key":"e_1_2_1_60_1","unstructured":"Sheriff's Justice Institute. 2016. Central Bond Court Report. (2016). https:\/\/www.chicagoreader.com\/pdf\/20161026\/Sheriff_s-Justice-Institute-Central-Bond-Court-Study-070616.pdf"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1037\/lhb0000360"},{"key":"e_1_2_1_62_1","first-page":"803","article-title":"Evidence-Based Sentencing and the Scientific Rationalization of Discrimination","volume":"66","author":"Starr Sonja B.","year":"2014","unstructured":"Sonja B. Starr. 2014. Evidence-Based Sentencing and the Scientific Rationalization of Discrimination. Stanford Law Review, Vol. 66, 4 (2014), 803--872. https:\/\/www.stanfordlawreview.org\/print\/article\/evidence-based-sentencing-and-the-scientific-rationalization-of-discrimination\/","journal-title":"Stanford Law Review"},{"key":"e_1_2_1_63_1","volume-title":"Juvenile detention risk assessment: A practice guide to juvenile detention reform","author":"Steinhart David","year":"2006","unstructured":"David Steinhart. 2006. Juvenile detention risk assessment: A practice guide to juvenile detention reform. The Annie E. Casey Foundation (2006). https:\/\/www.aecf.org\/m\/resourceimg\/aecf-juveniledetentionriskassessment1--2006.pdf"},{"key":"e_1_2_1_64_1","first-page":"303","article-title":"Assessing Risk Assessment in Action","volume":"103","author":"Stevenson Megan T.","year":"2018","unstructured":"Megan T. Stevenson. 2018. Assessing Risk Assessment in Action. Minnesota Law Review, Vol. 103 (2018), 303--384. https:\/\/scholarship.law.umn.edu\/mlr\/58\/","journal-title":"Minnesota Law Review"},{"key":"e_1_2_1_65_1","volume-title":"Doleac","author":"Stevenson Megan T.","year":"2021","unstructured":"Megan T. Stevenson and Jennifer L. Doleac. 2021. Algorithmic Risk Assessment in the Hands of Humans. (2021). https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=3489440"},{"key":"e_1_2_1_66_1","volume-title":"Leadership Conference Education Fund.","author":"The","year":"2018","unstructured":"The Leadership Conference Education Fund. 2018. The Use of Pretrial \"Risk Assessment\" Instruments: A Shared Statement of Civil Rights Concerns. (2018). https:\/\/leadershipconferenceedfund.org\/pretrial-risk-assessment\/"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.7455683"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.3886\/ICPSR02038.v5"},{"key":"e_1_2_1_69_1","unstructured":"USDA Rural Development. 2020. Single Family Housing Repair Loans & Grants. (2020). https:\/\/www.rd.usda.gov\/programs-services\/single-family-housing-repair-loans-grants"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287566"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2018.2890174"},{"key":"e_1_2_1_72_1","unstructured":"Human Rights Watch. 2017. \"Not in it for Justice\": How California's Pretrial Detention and Bail System Unfairly Punishes Poor People. (2017). https:\/\/www.hrw.org\/report\/2017\/04\/11\/not-it-justice\/how-californias-pretrial-detention-and-bail-system-unfairly"},{"key":"e_1_2_1_73_1","unstructured":"Wisconsin Supreme Court. 2016. State v. Loomis .881 Wis. N.W.2d 749."},{"key":"e_1_2_1_74_1","first-page":"1399","article-title":"Toward an Optimal Bail System","volume":"92","author":"Yang Crystal S.","year":"2017","unstructured":"Crystal S. Yang. 2017. Toward an Optimal Bail System. New York University Law Review, Vol. 92, 5 (2017), 1399--1493. https:\/\/www.nyulawreview.org\/issues\/volume-92-number-5\/toward-an-optimal-bail-system\/","journal-title":"New York University Law Review"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1002\/bdm.2118"},{"key":"e_1_2_1_76_1","volume-title":"When the State Meets the Street: Public Service and Moral Agency","author":"Zacka Bernardo","unstructured":"Bernardo Zacka. 2017. When the State Meets the Street: Public Service and Moral Agency .Harvard University Press."}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3479562","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3479562","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3479562","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T04:58:53Z","timestamp":1752469133000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3479562"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,13]]},"references-count":76,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2021,10,13]]}},"alternative-id":["10.1145\/3479562"],"URL":"https:\/\/doi.org\/10.1145\/3479562","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,13]]},"assertion":[{"value":"2021-10-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}