{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T15:06:57Z","timestamp":1787238417243,"version":"build-2736575974"},"reference-count":46,"publisher":"Springer Science and Business Media LLC","issue":"10","license":[{"start":{"date-parts":[[2022,7,4]],"date-time":"2022-07-04T00:00:00Z","timestamp":1656892800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,7,4]],"date-time":"2022-07-04T00:00:00Z","timestamp":1656892800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Nat Hum Behav"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Building artificial intelligence (AI) that aligns with human values is an unsolved problem. Here we developed a human-in-the-loop research pipeline called Democratic AI, in which reinforcement learning is used to design a social mechanism that humans prefer by majority. A large group of humans played an online investment game that involved deciding whether to keep a monetary endowment or to share it with others for collective benefit. Shared revenue was returned to players under two different redistribution mechanisms, one designed by the AI and the other by humans. The AI discovered a mechanism that redressed initial wealth imbalance, sanctioned free riders and successfully won the majority vote. By optimizing for human preferences, Democratic AI offers a proof of concept for value-aligned policy innovation.<\/jats:p>","DOI":"10.1038\/s41562-022-01383-x","type":"journal-article","created":{"date-parts":[[2022,7,4]],"date-time":"2022-07-04T12:03:30Z","timestamp":1656936210000},"page":"1398-1407","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":72,"title":["Human-centred mechanism design with Democratic AI"],"prefix":"10.1038","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5264-3137","authenticated-orcid":false,"given":"Raphael","family":"Koster","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jan","family":"Balaguer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"Tacchetti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ari","family":"Weinstein","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tina","family":"Zhu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9282-0801","authenticated-orcid":false,"given":"Oliver","family":"Hauser","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Duncan","family":"Williams","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lucy","family":"Campbell-Gillingham","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Phoebe","family":"Thacker","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7758-6896","authenticated-orcid":false,"given":"Matthew","family":"Botvinick","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2941-2653","authenticated-orcid":false,"given":"Christopher","family":"Summerfield","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,7,4]]},"reference":[{"key":"1383_CR1","doi-asserted-by":"publisher","first-page":"751","DOI":"10.1126\/science.aat5991","volume":"361","author":"M Taddeo","year":"2018","unstructured":"Taddeo, M. & Floridi, L. How AI can be a force for good. Science 361, 751\u2013752 (2018).","journal-title":"Science"},{"key":"1383_CR2","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1038\/s41586-021-03819-2","volume":"596","author":"J Jumper","year":"2021","unstructured":"Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583\u2013589 (2021).","journal-title":"Nature"},{"key":"1383_CR3","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1038\/s41586-019-1390-1","volume":"572","author":"N Tomasev","year":"2019","unstructured":"Tomasev, N. et al. A clinically applicable approach to continuous prediction of future acute kidney injury. Nature 572, 116\u2013119 (2019).","journal-title":"Nature"},{"key":"1383_CR4","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1145\/3339399","volume":"62","author":"C Gomes","year":"2019","unstructured":"Gomes, C. et al. Computational sustainability: computing for a better world and a sustainable future. Commun. ACM 62, 56\u201365 (2019).","journal-title":"Commun. ACM"},{"key":"1383_CR5","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1145\/3284751.3284761","volume":"4","author":"R Abebe","year":"2018","unstructured":"Abebe, R. & Goldner, K. Mechanism design for social good. AI Matters 4, 27\u201334 (2018).","journal-title":"AI Matters"},{"key":"1383_CR6","unstructured":"Roth, A. E. Who Gets What \u2014and Why: The New Economics of Matchmaking and Market Design (Houghton Mifflin Harcourt, 2016)."},{"key":"1383_CR7","doi-asserted-by":"publisher","unstructured":"Hurwicz, L. & Reiter, S. Designing Economic Mechanisms (Cambridge Univ. Press, 2006).https:\/\/doi.org\/10.1017\/CBO9780511754258","DOI":"10.1017\/CBO9780511754258"},{"key":"1383_CR8","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1007\/s11023-020-09539-2","volume":"30","author":"I Gabriel","year":"2020","unstructured":"Gabriel, I. Artificial intelligence, values and alignment. Minds Mach. 30, 411\u2013437 (2020).","journal-title":"Minds Mach."},{"key":"1383_CR9","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/s10676-017-9430-8","volume":"20","author":"I Rahwan","year":"2018","unstructured":"Rahwan, I. Society-in-the-loop: programming the algorithmic social contract. Ethics Inf. Technol. 20, 5\u201314 (2018).","journal-title":"Ethics Inf. Technol."},{"key":"1383_CR10","unstructured":"Doshi-Velez, F. & Kim, B. Towards a rigorous science of interpretable machine learning. Preprint at https:\/\/arxiv.org\/abs\/1702.08608 (2017)."},{"key":"1383_CR11","first-page":"1","volume":"3","author":"MK Lee","year":"2019","unstructured":"Lee, M. K. et al. WeBuildAI: participatory framework for algorithmic governance. Proc. ACM Hum. Comput. Interact. 3, 1\u201335 (2019).","journal-title":"Proc. ACM Hum. Comput. Interact."},{"key":"1383_CR12","doi-asserted-by":"crossref","unstructured":"Conitzer, V., Sinott-Armstrong, W., Borg, J. S. & Deng, Y. Moral Decision Making Frameworks for Artificial Intelligence (AAAI Press, 2017).","DOI":"10.1609\/aaai.v31i1.11140"},{"key":"1383_CR13","unstructured":"Dafoe, A. et al. Open problems in cooperative AI. Preprint at https:\/\/arxiv.org\/abs\/2012.08630 (2020)."},{"key":"1383_CR14","unstructured":"Christiano, P. et al. Deep reinforcement learning from human preferences. Preprint at https:\/\/arxiv.org\/abs\/1706.03741 (2017)."},{"key":"1383_CR15","doi-asserted-by":"publisher","unstructured":"Noothigattu, R. et al. A voting-based system for ethical decision making. Preprint at https:\/\/doi.org\/10.48550\/arXiv.1709.06692 (2018).","DOI":"10.48550\/arXiv.1709.06692"},{"key":"1383_CR16","unstructured":"Olson, M. The Logic of Collective Action: Public Goods and the Theory of Groups (Harvard Univ. Press, 2003)."},{"key":"1383_CR17","doi-asserted-by":"crossref","unstructured":"Ostrom, E. Governing the Commons: the Evolution of Institutions for Collective Action (Cambridge Univ. Press, 1991).","DOI":"10.1017\/CBO9780511807763"},{"key":"1383_CR18","unstructured":"Rawls, J. A Theory of Justice (Oxford Univ. Press, 2000)."},{"key":"1383_CR19","doi-asserted-by":"crossref","unstructured":"von Platz, J. Theories of Distributive Justice: Who Gets What and Why (Routledge, 2020).","DOI":"10.4324\/9780429318788"},{"key":"1383_CR20","doi-asserted-by":"publisher","unstructured":"Dworczak, P., Kominers, S. D. & Akbarpour, M. Redistribution Through Markets Becker Friedman Institute for Research in Economics Working Paper No. 2018-16 (SSRN, 2018); https:\/\/doi.org\/10.2139\/ssrn.3143887","DOI":"10.2139\/ssrn.3143887"},{"key":"1383_CR21","unstructured":"Nozick, R. Anarchy, State, and Utopia (Basic Books, 1974)."},{"key":"1383_CR22","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1016\/0047-2727(85)90038-6","volume":"26","author":"RM Isaac","year":"1985","unstructured":"Isaac, R. M., McCue, K. F. & Plott, C. R. Public goods provision in an experimental environment. J. Public Econ. 26, 51\u201374 (1985).","journal-title":"J. Public Econ."},{"key":"1383_CR23","doi-asserted-by":"publisher","first-page":"458","DOI":"10.1038\/s41562-018-0385-5","volume":"2","author":"E Fehr","year":"2018","unstructured":"Fehr, E. & Schurtenberger, I. Normative foundations of human cooperation. Nat. Hum. Behav. 2, 458\u2013468 (2018).","journal-title":"Nat. Hum. Behav."},{"key":"1383_CR24","doi-asserted-by":"publisher","first-page":"209","DOI":"10.5840\/soctheorpract1979523","volume":"5","author":"K Nielsen","year":"1979","unstructured":"Nielsen, K. Radical egalitarian justice: justice as equality. Soc. Theory Pract. 5, 209\u2013226 (1979).","journal-title":"Soc. Theory Pract."},{"key":"1383_CR25","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1257\/aer.90.1.247","volume":"90","author":"J Falkinger","year":"2000","unstructured":"Falkinger, J., Fehr, E., G\u00e4chter, S. & Winter-Ebmer, R. A simple mechanism for the efficient provision of public goods: experimental evidence. Am. Econ. Rev. 90, 247\u2013264 (2000).","journal-title":"Am. Econ. Rev."},{"key":"1383_CR26","doi-asserted-by":"crossref","unstructured":"Dworkin, R. Sovereign Virtue: The Theory and Practice of Equality (Harvard Univ. Press, 2002).","DOI":"10.2307\/j.ctv1c3pd0r"},{"key":"1383_CR27","doi-asserted-by":"publisher","first-page":"935","DOI":"10.1016\/j.jpubeco.2005.06.002","volume":"90","author":"E Buckley","year":"2006","unstructured":"Buckley, E. & Croson, R. Income and wealth heterogeneity in the voluntary provision of linear public goods. J. Public Econ. 90, 935\u2013955 (2006).","journal-title":"J. Public Econ."},{"key":"1383_CR28","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1016\/j.jebo.2003.11.010","volume":"57","author":"TL Cherry","year":"2005","unstructured":"Cherry, T. L., Kroll, S. & Shogren, J. F. The impact of endowment heterogeneity and origin on public good contributions: evidence from the lab. J. Econ. Behav. Organ. 57, 357\u2013365 (2005).","journal-title":"J. Econ. Behav. Organ."},{"key":"1383_CR29","doi-asserted-by":"publisher","first-page":"101679","DOI":"10.1016\/j.socec.2021.101679","volume":"92","author":"P Bra\u00f1as-Garza","year":"2021","unstructured":"Bra\u00f1as-Garza, P., Molis, E. & Neyse, L. Exposure to inequality may cause under-provision of public goods: experimental evidence. J. Behav. Exp. Econ. 92, 101679 (2021).","journal-title":"J. Behav. Exp. Econ."},{"key":"1383_CR30","first-page":"3528","volume":"28","author":"J Schulman","year":"2015","unstructured":"Schulman, J., Heess, N., Weber, T. & Abbeel, P. Gradient estimation using stochastic computation graphs. NeurIPS 28, 3528\u20133536 (2015).","journal-title":"NeurIPS"},{"key":"1383_CR31","doi-asserted-by":"publisher","unstructured":"Carroll, M. et al. On the utility of learning about humans for human-AI coordination. Preprint at https:\/\/doi.org\/10.48550\/arXiv.1910.05789 (2020).","DOI":"10.48550\/arXiv.1910.05789"},{"key":"1383_CR32","doi-asserted-by":"publisher","first-page":"29221","DOI":"10.1073\/pnas.2016921117","volume":"117","author":"A Dezfouli","year":"2020","unstructured":"Dezfouli, A., Nock, R. & Dayan, P. Adversarial vulnerabilities of human decision-making. Proc. Natl Acad. Sci. USA 117, 29221\u201329228 (2020).","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"1383_CR33","doi-asserted-by":"publisher","unstructured":"Abramson, J. et al. Imitating interactive intelligence. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2012.05672 (2021).","DOI":"10.48550\/arXiv.2012.05672"},{"key":"1383_CR34","first-page":"14502","volume":"34","author":"DJ Strouse","year":"2021","unstructured":"Strouse, D. J., McKee, K., Botvinick, M. M., Hughes, E. & Everett, R. Collaborating with Humans without Human Data. NeurIPS 34, 14502\u201314515 (2021).","journal-title":"NeurIPS"},{"key":"1383_CR35","doi-asserted-by":"publisher","unstructured":"Zheng, S. et al. The AI economist: improving equality and productivity with AI-driven tax policies. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2004.13332 (2020).","DOI":"10.48550\/arXiv.2004.13332"},{"key":"1383_CR36","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1126\/science.1123633","volume":"312","author":"O Gurerk","year":"2006","unstructured":"Gurerk, O. The competitive advantage of sanctioning institutions. Science 312, 108\u2013111 (2006).","journal-title":"Science"},{"key":"1383_CR37","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1007\/s10683-019-09609-y","volume":"23","author":"A Dannenberg","year":"2020","unstructured":"Dannenberg, A., Haita-Falah, C. & Zitzelsberger, S. Voting on the threat of exclusion in a public goods experiment. Exp. Econ. 23, 84\u2013109 (2020).","journal-title":"Exp. Econ."},{"key":"1383_CR38","doi-asserted-by":"publisher","first-page":"693","DOI":"10.2307\/2297769","volume":"50","author":"RJ Aumann","year":"1983","unstructured":"Aumann, R. J., Kurz, M. & Neyman, A. Voting for public goods. Rev. Econ. Stud. 50, 693 (1983).","journal-title":"Rev. Econ. Stud."},{"key":"1383_CR39","first-page":"34","volume":"64","author":"J-R Tyran","year":"2010","unstructured":"Tyran, J.-R., Rauchdobler, J. & Sausgruber, R. Voting on thresholds for public goods: experimental evidence. FinanzArchiv 64, 34 (2010).","journal-title":"FinanzArchiv"},{"key":"1383_CR40","unstructured":"Beahm, D. L. Conceptions of and Corrections to Majoritarian Tyranny (Lexington Books, 2002)."},{"key":"1383_CR41","doi-asserted-by":"publisher","first-page":"659","DOI":"10.1007\/s13347-020-00405-8","volume":"33","author":"S Mohamed","year":"2020","unstructured":"Mohamed, S., Png, M.-T. & Isaac, W. Decolonial AI: decolonial theory as sociotechnical foresight in artificial intelligence. Phil. Technol. 33, 659\u2013684 (2020).","journal-title":"Phil. Technol."},{"key":"1383_CR42","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-87480-9","volume":"11","author":"E Bogert","year":"2021","unstructured":"Bogert, E., Schecter, A. & Watson, R. T. Humans rely more on algorithms than social influence as a task becomes more difficult. Sci. Rep. 11, 8028 (2021).","journal-title":"Sci. Rep."},{"key":"1383_CR43","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1016\/j.tics.2009.09.004","volume":"13","author":"R Hertwig","year":"2009","unstructured":"Hertwig, R. & Erev, I. The description-experience gap in risky choice. Trends Cogn. Sci. 13, 517\u2013523 (2009).","journal-title":"Trends Cogn. Sci."},{"key":"1383_CR44","doi-asserted-by":"publisher","unstructured":"Katzenbach, C. & Ulbricht, L. Algorithmic governance. Internet Policy Rev. https:\/\/doi.org\/10.14763\/2019.4.1424 (2019).","DOI":"10.14763\/2019.4.1424"},{"key":"1383_CR45","doi-asserted-by":"publisher","first-page":"101283","DOI":"10.1016\/j.techsoc.2020.101283","volume":"62","author":"HS S\u00e6tra","year":"2020","unstructured":"S\u00e6tra, H. S. A shallow defence of a technocracy of artificial intelligence: examining the political harms of algorithmic governance in the domain of government. Technol. Soc. 62, 101283 (2020).","journal-title":"Technol. Soc."},{"key":"1383_CR46","unstructured":"Peters, B. G. The Politics of Bureaucracy: an Introduction to Comparative Public Administration (Routledge, 2010)."}],"updated-by":[{"DOI":"10.1038\/s41562-022-01444-1","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2022,8,12]],"date-time":"2022-08-12T00:00:00Z","timestamp":1660262400000}}],"container-title":["Nature Human Behaviour"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41562-022-01383-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41562-022-01383-x","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41562-022-01383-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,24]],"date-time":"2022-11-24T10:01:32Z","timestamp":1669284092000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41562-022-01383-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,4]]},"references-count":46,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["1383"],"URL":"https:\/\/doi.org\/10.1038\/s41562-022-01383-x","relation":{},"ISSN":["2397-3374"],"issn-type":[{"value":"2397-3374","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,4]]},"assertion":[{"value":"9 December 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 May 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 July 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 August 2022","order":4,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Correction","order":5,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A Correction to this paper has been published:","order":6,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"https:\/\/doi.org\/10.1038\/s41562-022-01444-1","URL":"https:\/\/doi.org\/10.1038\/s41562-022-01444-1","order":7,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}