{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,2]],"date-time":"2026-08-02T11:51:29Z","timestamp":1785671489134,"version":"3.56.0"},"reference-count":49,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2019,5,8]],"date-time":"2019-05-08T00:00:00Z","timestamp":1557273600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>This essay argues that a new subfield of AI governance should be explored that examines the policy-making process and its implications for AI governance. A growing number of researchers have begun working on the question of how to mitigate the catastrophic risks of transformative artificial intelligence, including what policies states should adopt. However, this essay identifies a preceding, meta-level problem of how the space of possible policies is affected by the politics and administrative mechanisms of how those policies are created and implemented. This creates a new set of key considerations for the field of AI governance and should influence the action of future policymakers. This essay examines some of the theories of the policymaking process, how they compare to current work in AI governance, and their implications for the field at large and ends by identifying areas of future research.<\/jats:p>","DOI":"10.3390\/bdcc3020026","type":"journal-article","created":{"date-parts":[[2019,5,13]],"date-time":"2019-05-13T03:57:07Z","timestamp":1557719827000},"page":"26","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":49,"title":["AI Governance and the Policymaking Process: Key Considerations for Reducing AI Risk"],"prefix":"10.3390","volume":"3","author":[{"given":"Brandon","family":"Perry","sequence":"first","affiliation":[{"name":"Independent Researcher, Berkeley, CA 94709, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2933-4608","authenticated-orcid":false,"given":"Risto","family":"Uuk","sequence":"additional","affiliation":[{"name":"Effective Altruism Estonia, 12618 Tallinn, Estonia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,5,8]]},"reference":[{"key":"ref_1","unstructured":"Tegmark, M. (2017). Life 3.0: Being Human in the Age of Artificial Intelligence, Knopf."},{"key":"ref_2","unstructured":"Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies, Oxford University Press."},{"key":"ref_3","unstructured":"Dafoe, A. (2018). AI Governance: A Research Agenda, Governance of AI Program, Future of Humanity Institute. Available online: https:\/\/www.fhi.ox.ac.uk\/govaiagenda\/."},{"key":"ref_4","unstructured":"Baum, S.D. (2019, November 11). A Survey of Artificial General Intelligence Projects for Ethics, Risk, and Policy. Global Catastrophic Risk Institute Working Paper 17-1. Available online: https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=3070741."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Everitt, T., Lea, G., and Hutter, M. (2018). AGI Safety Literature Review. arXiv.","DOI":"10.24963\/ijcai.2018\/768"},{"key":"ref_6","first-page":"238","article-title":"Why the future doesn\u2019t need us","volume":"8","author":"Joy","year":"2000","journal-title":"Wired"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hibbard, B. (2002). Super-Intelligent Machines, Springer.","DOI":"10.1007\/978-1-4615-0759-8"},{"key":"ref_8","unstructured":"Allhoff, F. (2007). Global technology regulation and potentially apocalyptic technological threats. Nanoethics: The Ethical and Social Implications of Nanotechnology, John Wiley."},{"key":"ref_9","first-page":"366","article-title":"Accelerating AI","volume":"104","author":"McGinnis","year":"2010","journal-title":"Northwest. Univ. Law Rev."},{"key":"ref_10","first-page":"354","article-title":"Regulating artificial intelligence systems: Risks, challenges, competencies, and strategies","volume":"29","author":"Scherer","year":"2016","journal-title":"Harv. J. Law Technol."},{"key":"ref_11","first-page":"385","article-title":"Nudging robots: Innovative solutions to regulate artificial intelligence","volume":"20","author":"Guihot","year":"2017","journal-title":"Vanderbilt J. Entertain. Technol. Law"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1007\/s00146-016-0677-0","article-title":"On the promotion of safe and socially beneficial artificial intelligence","volume":"32","author":"Baum","year":"2017","journal-title":"AI Soc."},{"key":"ref_13","first-page":"217","article-title":"Safety Engineering for Artificial General Intelligence","volume":"32","author":"Yampolskiy","year":"2013","journal-title":"Topoi"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Erdelyi, O.J., and Goldsmith, J. (2018, January 2\u20133). Regulating Artificial Intelligence: Proposal for a Global Solution. Proceedings of the 2018 AAAI\/ACM Conference on AI, Ethics, and Society (AIES \u201918), New Orleans, LO, USA. Available online: https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=3263992.","DOI":"10.1145\/3278721.3278731"},{"key":"ref_15","first-page":"307","article-title":"Minimizing global catastrophic and existential risks from emerging technologies through international law","volume":"31","author":"Wilson","year":"2013","journal-title":"Va. Environ. Law J."},{"key":"ref_16","unstructured":"Shulman, C. (2009, January 2\u20134). Arms control and intelligence explosions. Proceedings of the 7th European Conference on Computing and Philosophy (ECAP), Bellaterra, Spain."},{"key":"ref_17","unstructured":"Goertzel, B. (2019, January 06). The Corporatization of AI is a Major Threat to Humanity. h+ Magazine. Available online: http:\/\/hplusmagazine.com\/2017\/07\/21\/corporatization-ai-major-threat-humanity\/."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1111\/1758-5899.12403","article-title":"Strategic Implications of Openness in AI Development","volume":"8","author":"Bostrom","year":"2017","journal-title":"Glob. Policy"},{"key":"ref_19","unstructured":"M\u00fcller, V.C. (2015). Long-term strategies for ending existential risk from fast takeoff. Risks of Artificial Intelligence, CRC."},{"key":"ref_20","first-page":"96","article-title":"Should Humanity Build a Global AI Nanny to Delay the Singularity Until It\u2019s Better Understood?","volume":"19","author":"Goertzel","year":"2012","journal-title":"J. Conscious. Stud."},{"key":"ref_21","unstructured":"Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., and Filar, B. (2018, January 06). The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation. Available online: https:\/\/img1.wsimg.com\/blobby\/go\/3d82daa4-97fe-4096-9c6b-376b92c619de\/downloads\/1c6q2kc4v_50335.pdf."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Whittlestone, J., Nyrup, R., Alexandrova, A., and Cave, S. (2019, January 27\u201328). The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions. Proceedings of the AAAI\/ACM Conference on AI Ethics and Society, Honolulu, HI, USA.","DOI":"10.1145\/3306618.3314289"},{"key":"ref_23","unstructured":"Calo, R. (2019, January 06). Artificial Intelligence Policy: A Primer and Roadmap. It should also be noted that Calo is dismissive of the risk of artificial general intelligence."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Cave, S., and \u00d3h\u00c9igeartaigh, S.S. (2019, January 27\u201328). An AI Race for Strategic Advantage: Rhetoric and Risks. Proceedings of the 2018 AAAI\/ACM Conference on AI, Ethics, and Society, Honolulu, HI, USA. Available online: http:\/\/www.aies-conference.com\/wp-content\/papers\/main\/AIES_2018_paper_163.pdf.","DOI":"10.1145\/3278721.3278780"},{"key":"ref_25","unstructured":"Flynn, C. (2019, January 06). Personal Thoughts on Careers in AI Policy and Strategy. Effective Altruism Forum. Available online: https:\/\/forum.effectivealtruism.org\/posts\/RCvetzfDnBNFX7pLH\/personal-thoughts-on-careers-in-ai-policy-and-strategy."},{"key":"ref_26","unstructured":"The specifics issues will depend on the type of government. For example, the types of difficulties would be different in a democracy vs. a dictatorship. This paper however will focus on federal republics."},{"key":"ref_27","unstructured":"Thank you to Sabrina Kavanagh for suggesting the idea that the policy process could inspire new ideas for AI governance researchers."},{"key":"ref_28","unstructured":"Brundage, M., and Bryson, J. (2016). Smart Policies for Artificial Intelligence. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"298","DOI":"10.2307\/974990","article-title":"Four Systems of Policy, Politics, and Choice","volume":"32","author":"Lowi","year":"1972","journal-title":"Public Adm. Rev."},{"key":"ref_30","unstructured":"Anderson, J.E. (2010). Public Policymaking: An Introduction, Cengage Learning. [7th ed.]."},{"key":"ref_31","unstructured":"Sabatier, P. (2007). The Multiple Streams Framework: Structure, Limitations, Prospects. Theories of the Policy Process, Westview Press. [2nd ed.]."},{"key":"ref_32","unstructured":"Cobb, R., and Elder, C.D. (1983). What is an Issue? What Makes an Issue?. Participation in American Politics: The Dynamics of Agenda Building, Johns Hopkins University Press."},{"key":"ref_33","unstructured":"Allen, G. (2019, February 26). China\u2019s Artificial Intelligence Strategy Poses a Credible Threat to U.S. Tech Leadership. Center for Foreign Affairs Blog. Available online: https:\/\/www.cfr.org\/blog\/chinas-artificial-intelligence-strategy-poses-credible-threat-us-tech-leadership."},{"key":"ref_34","unstructured":"Yampolskiy, R. (2018, January 06). Current State of Knowledge on Failures of AI Enabled Products. Report. Consortium for Safer AI. Available online: https:\/\/docs.wixstatic.com\/ugd\/ace275_0ea60fe9b665439bb0b37d20beb89b6f.pdf."},{"key":"ref_35","unstructured":"Danzig, R. (2018). Technology Roulette: Managing Loss of Control as Many Militaries Pursue Technological Superiority, Center for New American Security. Available online: https:\/\/www.cnas.org\/publications\/reports\/technology-roulette."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1017\/S0143814X0000619X","article-title":"Reconsidering Policy Design: Policies and Publics","volume":"11","author":"May","year":"1991","journal-title":"J. Public Policy"},{"key":"ref_37","unstructured":"Russell, S., Aguirre, A., Conn, A., and Tegmark, M. (2019, January 09). Why You Should Fear \u201cSlaughterbots\u201d\u2014A Response. IEEE Spectrum. Available online: https:\/\/spectrum.ieee.org\/automaton\/robotics\/artificial-intelligence\/why-you-should-fear-slaughterbots-a-response."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Bostrom, N., and \u0106irkovi\u0107, M.M. (2008). Cognitive Biases Potentially Affecting Judgment of Global Risks. Global Catastrophic Risks, Oxford University Press.","DOI":"10.1093\/oso\/9780198570509.001.0001"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Baum, S.D. (2018). Superintelligence Skepticism as a Political Tool. Information, 9.","DOI":"10.3390\/info9090209"},{"key":"ref_40","unstructured":"Sabatier, P.A. (2007). Punctuated-Equilibrium Theory: Explaining Stability and Change in Public Policymaking. Theories of the Policy Process, Westview Press. [2nd ed.]. Chapter 6."},{"key":"ref_41","unstructured":"Sabatier, P.A. (2007). An Advocacy Coalition Framework. Theories of the Policy Process, Westview Press. [2nd ed.]. Chapter 7."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"171","DOI":"10.3102\/01623737009002171","article-title":"Learning From Experience: Lessons From Policy Implementation","volume":"9","author":"McLaughlin","year":"1987","journal-title":"Educ. Eval. Policy Anal."},{"key":"ref_43","unstructured":"Farquhar, S. (2019, January 06). Changes in Funding in the AI Safety Field. Available online: https:\/\/www.centreforeffectivealtruism.org\/blog\/changes-in-funding-in-the-ai-safety-field."},{"key":"ref_44","unstructured":"MacAskill, W. (2019, January 06). What Are the Most Important Moral Problems of Our Time?. Available online: https:\/\/www.ted.com\/talks\/will_macaskill_how_can_we_do_the_most_good_for_the_world."},{"key":"ref_45","unstructured":"M\u00fcller, V.C. (2014). Future progress in artificial intelligence: A Survey of Expert Opinion. Fundamental Issues of Artificial Intelligence, Springer. Available online: https:\/\/nickbostrom.com\/papers\/survey.pdf."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Grace, K., Salvatier, J., Dafoe, A., Zhang, B., and Evans, O. (2017). When Will AI Exceed Human Performance? Evidence from AI Experts. arXiv.","DOI":"10.1613\/jair.1.11222"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Zhang, B., and Dafoe, A. (2019, January 03). Artificial Intelligence: American Attitudes and Trends. Available online: https:\/\/governanceai.github.io\/US-Public-Opinion-Report-Jan-2019\/us_public_opinion_report_jan_2019.pdf.","DOI":"10.2139\/ssrn.3312874"},{"key":"ref_48","first-page":"481","article-title":"The Conditions of Effective Implementation: A Guide to Accomplishing Policy Objectives","volume":"5","author":"Sabatier","year":"1979","journal-title":"Policy Anal."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"538","DOI":"10.1111\/j.1541-0072.1980.tb01266.x","article-title":"The Implementation of Public Policy: A Framework of Analysis","volume":"8","author":"Sabatier","year":"1980","journal-title":"Policy Stud. J."}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/3\/2\/26\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:50:09Z","timestamp":1760187009000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/3\/2\/26"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,8]]},"references-count":49,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2019,6]]}},"alternative-id":["bdcc3020026"],"URL":"https:\/\/doi.org\/10.3390\/bdcc3020026","relation":{},"ISSN":["2504-2289"],"issn-type":[{"value":"2504-2289","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,5,8]]}}}