{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T21:54:56Z","timestamp":1776117296847,"version":"3.50.1"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2024,11,7]],"date-time":"2024-11-07T00:00:00Z","timestamp":1730937600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-sa\/4.0\/"}],"funder":[{"name":"Digital Transformation and Innovation Center"},{"DOI":"10.13039\/501100006374","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1952085, IIS2040929, IIS222988"],"award-info":[{"award-number":["1952085, IIS2040929, IIS222988"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]},{"name":"K&L Gates Presidential Fellowship"},{"name":"UL Research Institutes"},{"DOI":"10.13039\/100023581","name":"NSF Graduate Research Fellowship Program","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100023581","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2024,11,7]]},"abstract":"<jats:p>As public sector agencies rapidly introduce new AI tools in high-stakes domains like social services, it becomes critical to understand how decisions to adopt these tools are made in practice. We borrow from the anthropological practice to \"study up\" those in positions of power, and reorient our study of public sector AI around those who have the power and responsibility to make decisions about the role that AI tools will play in their agency. Through semi-structured interviews and design activities with 16 agency decision-makers, we examine how decisions about AI design and adoption are influenced by their interactions with and assumptions about other actors within these agencies (e.g., frontline workers and agency leaders), as well as those above (legal systems and contracted companies), and below (impacted communities). By centering these networks of power relations, our findings shed light on how infrastructural, legal, and social factors create barriers and disincentives to the involvement of a broader range of stakeholders in decisions about AI design and adoption. Agency decision-makers desired more practical support for stakeholder involvement around public sector AI to help overcome the knowledge and power differentials they perceived between them and other stakeholders (e.g., frontline workers and impacted community members). Building on these findings, we discuss implications for future research and policy around actualizing participatory AI approaches in public sector contexts.<\/jats:p>","DOI":"10.1145\/3686989","type":"journal-article","created":{"date-parts":[[2024,11,8]],"date-time":"2024-11-08T15:52:40Z","timestamp":1731081160000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":18,"title":["Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-6937-6370","authenticated-orcid":false,"given":"Anna","family":"Kawakami","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9282-9921","authenticated-orcid":false,"given":"Amanda","family":"Coston","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3710-4076","authenticated-orcid":false,"given":"Hoda","family":"Heidari","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6730-922X","authenticated-orcid":false,"given":"Kenneth","family":"Holstein","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7271-9100","authenticated-orcid":false,"given":"Haiyi","family":"Zhu","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,11,8]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1080\/01944366908977225"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v7i1.5285"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372859"},{"key":"e_1_2_1_4_1","unstructured":"Solon Barocas Moritz Hardt and Arvind Narayanan. 2019. Fairness and Machine Learning. fairmlbook.org. http:\/\/www.fairmlbook.org."},{"key":"e_1_2_1_5_1","volume-title":"Contextual design. interactions","author":"Beyer Hugh","year":"1999","unstructured":"Hugh Beyer and Karen Holtzblatt. 1999. Contextual design. interactions, Vol. 6, 1 (1999), 32--42."},{"key":"e_1_2_1_6_1","volume-title":"Reflecting on reflexive thematic analysis. Qualitative research in sport, exercise and health","author":"Braun Virginia","year":"2019","unstructured":"Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative research in sport, exercise and health, Vol. 11, 4 (2019), 589--597."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300271"},{"key":"e_1_2_1_8_1","volume-title":"How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions. In CHI Conference on Human Factors in Computing Systems. 1--22","author":"Cheng Hao-Fei","year":"2022","unstructured":"Hao-Fei Cheng, Logan Stapleton, Anna Kawakami, Venkatesh Sivaraman, Yanghuidi Cheng, Diana Qing, Adam Perer, Kenneth Holstein, Zhiwei Steven Wu, and Haiyi Zhu. 2022. How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions. In CHI Conference on Human Factors in Computing Systems. 1--22."},{"key":"e_1_2_1_9_1","volume-title":"Conference on Fairness, Accountability and Transparency. PMLR, 134--148","author":"Chouldechova Alexandra","year":"2018","unstructured":"Alexandra Chouldechova, Diana Benavides-Prado, Oleksandr Fialko, and Rhema Vaithianathan. 2018. A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In Conference on Fairness, Accountability and Transparency. PMLR, 134--148."},{"key":"e_1_2_1_10_1","volume-title":"A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms. arXiv preprint arXiv:2206.14983","author":"Coston Amanda","year":"2022","unstructured":"Amanda Coston, Anna Kawakami, Haiyi Zhu, Ken Holstein, and Hoda Heidari. 2022. A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms. arXiv preprint arXiv:2206.14983 (2022)."},{"key":"e_1_2_1_11_1","volume-title":"The atlas of AI: Power, politics, and the planetary costs of artificial intelligence","author":"Crawford Kate","unstructured":"Kate Crawford. 2021. The atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press."},{"key":"e_1_2_1_12_1","volume-title":"Stakeholder Participation in AI: Beyond\" Add Diverse Stakeholders and Stir\". arXiv preprint arXiv:2111.01122","author":"Delgado Fernando","year":"2021","unstructured":"Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2021. Stakeholder Participation in AI: Beyond\" Add Diverse Stakeholders and Stir\". arXiv preprint arXiv:2111.01122 (2021)."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174016"},{"key":"e_1_2_1_14_1","volume-title":"Automating inequality: How high-tech tools profile, police, and punish the poor","author":"Eubanks Virginia","unstructured":"Virginia Eubanks. 2018. Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin's Press."},{"key":"e_1_2_1_15_1","volume-title":"Studying those who study us: An anthropologist in the world of artificial intelligence","author":"Forsythe Diana","unstructured":"Diana Forsythe. 2001. Studying those who study us: An anthropologist in the world of artificial intelligence. Stanford University Press."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3593013.3594081"},{"key":"e_1_2_1_17_1","volume-title":"Impact evaluation of a predictive risk modeling tool for Allegheny county's child welfare office","author":"Goldhaber-Fiebert Jeremy D","year":"2019","unstructured":"Jeremy D Goldhaber-Fiebert and Lea Prince. 2019. Impact evaluation of a predictive risk modeling tool for Allegheny county's child welfare office. Pittsburgh: Allegheny County (2019)."},{"key":"e_1_2_1_18_1","volume-title":"Companion proceedings of the ninth international learning analytics & knowledge conference (LAK","author":"Holstein Kenneth","year":"2019","unstructured":"Kenneth Holstein and Shayan Doroudi. 2019. Fairness and equity in learning analytics systems (FairLAK). In Companion proceedings of the ninth international learning analytics & knowledge conference (LAK 2019). 1--2."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3027385.3027451"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-93843-1_12"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3419249.3420149"},{"key":"e_1_2_1_22_1","unstructured":"White House. 2023. Fact sheet: President Biden issues executive order on safe secure and trustworthy artificial intelligence."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3532106.3533564"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642849"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3517439"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3532106.3533556"},{"key":"e_1_2_1_27_1","unstructured":"Rob Kitchin and Tracey Lauriault. 2014. Towards critical data studies: Charting and unpacking data assemblages and their work. (2014)."},{"key":"e_1_2_1_28_1","volume-title":"International Conference on Machine Learning Workshop","volume":"7","author":"Kulynych Bogdan","year":"2020","unstructured":"Bogdan Kulynych, David Madras, Smitha Milli, Inioluwa Deborah Raji, Angela Zhou, and Richard Zemel. 2020. Participatory approaches to machine learning. In International Conference on Machine Learning Workshop, Vol. 7."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580882"},{"key":"e_1_2_1_30_1","volume-title":"AI governance in the public sector: Three tales from the frontiers of automated decision-making in democratic settings. Telecommunications policy","author":"Kuziemski Maciej","year":"2020","unstructured":"Maciej Kuziemski and Gianluca Misuraca. 2020. AI governance in the public sector: Three tales from the frontiers of automated decision-making in democratic settings. Telecommunications policy, Vol. 44, 6 (2020), 101976."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-lawsocsci-041221-023808"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-lawsocsci-041221-023808"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3461702.3462631"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3492853"},{"key":"e_1_2_1_35_1","unstructured":"Laura Nader. 1972. Up the anthropologist: Perspectives gained from studying up. (1972)."},{"key":"e_1_2_1_36_1","volume-title":"Science","volume":"366","author":"Obermeyer Ziad","year":"2019","unstructured":"Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. Dissecting racial bias in an algorithm used to manage the health of populations. Science, Vol. 366, 6464 (2019), 447--453."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287567"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3152420"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533158"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445748"},{"key":"e_1_2_1_41_1","volume-title":"Family surveillance by algorithm: The rapidly spreading tools few have heard of","author":"Samant Anjana","year":"2021","unstructured":"Anjana Samant, Aaron Horowitz, Kath Xu, and Sophie Beiers. 2021. Family surveillance by algorithm: The rapidly spreading tools few have heard of. American Civil Liberties Union (ACLU)(2021)."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376229"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3476089"},{"key":"e_1_2_1_44_1","volume-title":"Ethnography Matters","volume":"10","author":"Seaver Nick","year":"2014","unstructured":"Nick Seaver. 2014. Studying up: The ethnography of technologists. Ethnography Matters, Vol. 10 (2014)."},{"key":"e_1_2_1_45_1","volume-title":"Nature","volume":"541","author":"Shapiro Aaron","year":"2017","unstructured":"Aaron Shapiro. 2017. Reform predictive policing. Nature, Vol. 541, 7638 (2017), 458--460."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3551624.3555285"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376783"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533177"},{"key":"e_1_2_1_49_1","volume-title":"Investigating human machine complementarity for recidivism predictions. arXiv preprint arXiv:1808.09123","author":"Tan Sarah","year":"2018","unstructured":"Sarah Tan, Julius Adebayo, Kori Inkpen, and Ece Kamar. 2018. Investigating human machine complementarity for recidivism predictions. arXiv preprint arXiv:1808.09123 (2018)."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2003.1236470"},{"key":"e_1_2_1_51_1","volume-title":"40th International Conference on Information Systems, ICIS 2019","author":"van den Broek Elmira","year":"2020","unstructured":"Elmira van den Broek, Anastasia Sergeeva, and Marleen Huysman. 2020. Hiring algorithms: An ethnography of fairness in practice. 40th International Conference on Information Systems, ICIS 2019 (2020)."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174014"},{"key":"e_1_2_1_53_1","volume-title":"Against predictive optimization: On the legitimacy of decision-making algorithms that optimize predictive accuracy. Available at SSRN","author":"Wang Angelina","year":"2022","unstructured":"Angelina Wang, Sayash Kapoor, Solon Barocas, and Arvind Narayanan. 2022. Against predictive optimization: On the legitimacy of decision-making algorithms that optimize predictive accuracy. Available at SSRN (2022)."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272973.3273005"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3579621"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300468"},{"key":"e_1_2_1_57_1","volume-title":"Andrew L Beam, and Isaac S Kohane","author":"Yu Kun-Hsing","year":"2018","unstructured":"Kun-Hsing Yu, Andrew L Beam, and Isaac S Kohane. 2018. Artificial intelligence in healthcare. Nature biomedical engineering, Vol. 2, 10 (2018), 719--731."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3579601"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274463"},{"key":"e_1_2_1_60_1","volume-title":"Sibyl: Understanding and addressing the usability challenges of machine learning in high-stakes decision making","author":"Zytek Alexandra","year":"2021","unstructured":"Alexandra Zytek, Dongyu Liu, Rhema Vaithianathan, and Kalyan Veeramachaneni. 2021. Sibyl: Understanding and addressing the usability challenges of machine learning in high-stakes decision making. IEEE Transactions on Visualization and Computer Graphics (2021)."}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3686989","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3686989","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3686989","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T00:50:11Z","timestamp":1755737411000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3686989"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,7]]},"references-count":60,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2024,11,7]]}},"alternative-id":["10.1145\/3686989"],"URL":"https:\/\/doi.org\/10.1145\/3686989","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,7]]},"assertion":[{"value":"2024-11-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}