{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T22:45:39Z","timestamp":1783550739461,"version":"3.55.0"},"reference-count":43,"publisher":"National Academy of Sciences","issue":"46","license":[{"start":{"date-parts":[[2023,11,6]],"date-time":"2023-11-06T00:00:00Z","timestamp":1699228800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["www.pnas.org"],"crossmark-restriction":true},"short-container-title":["Proc. Natl. Acad. Sci. U.S.A."],"published-print":{"date-parts":[[2023,11,14]]},"abstract":"<jats:p>Collective intelligence challenges are often entangled with collective action problems. For example, voting, rating, and social innovation are collective intelligence tasks that require costly individual contributions. As a result, members of a group often free ride on the information contributed by intrinsically motivated people. Are intrinsically motivated agents the best participants in collective decisions? We embedded a collective intelligence task in a large-scale, virtual world public good game and found that participants who joined the information system but were reluctant to contribute to the public good (free riders) provided more accurate evaluations, whereas participants who rated frequently underperformed. Testing the underlying mechanism revealed that a negative rating bias in free riders is associated with higher accuracy. Importantly, incentivizing evaluations amplifies the relative influence of participants who tend to free ride without altering the (higher) quality of their evaluations, thereby improving collective intelligence. These results suggest that many of the currently available information systems, which strongly select for intrinsically motivated participants, underperform and that collective intelligence can benefit from incentivizing free riding members to engage. More generally, enhancing the diversity of contributor motivations can improve collective intelligence in settings that are entangled with collective action problems.<\/jats:p>","DOI":"10.1073\/pnas.2311497120","type":"journal-article","created":{"date-parts":[[2023,11,6]],"date-time":"2023-11-06T15:30:36Z","timestamp":1699284636000},"update-policy":"https:\/\/doi.org\/10.1073\/pnas.cm10313","source":"Crossref","is-referenced-by-count":4,"title":["Incentivizing free riders improves collective intelligence in social dilemmas"],"prefix":"10.1073","volume":"120","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6788-614X","authenticated-orcid":false,"given":"Ofer","family":"Tchernichovski","sequence":"first","affiliation":[{"name":"Department of Psychology, Hunter College, The City University of New York, New York, NY 10065"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5298-5089","authenticated-orcid":false,"given":"Seth","family":"Frey","sequence":"additional","affiliation":[{"name":"Department of Communication, University of California, Davis, CA 95616"},{"name":"Ostrom Workshop, Indiana University Bloomington, Bloomington, IN 47408"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2868-0100","authenticated-orcid":false,"given":"Nori","family":"Jacoby","sequence":"additional","affiliation":[{"name":"Computational Auditory Perception Group, Max Planck Institute for Empirical Aesthetics, Frankfurt 60322, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5174-7222","authenticated-orcid":false,"given":"Dalton","family":"Conley","sequence":"additional","affiliation":[{"name":"Princeton and National Bureau of Economic Research, Department of Sociology and Office of Population Research, Princeton University, Princeton, NJ 08544"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"341","published-online":{"date-parts":[[2023,11,6]]},"reference":[{"key":"e_1_3_4_1_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1615978114"},{"key":"e_1_3_4_2_2","doi-asserted-by":"publisher","DOI":"10.1038\/ncomms13800"},{"key":"e_1_3_4_3_2","doi-asserted-by":"publisher","DOI":"10.1177\/26339137221083293"},{"key":"e_1_3_4_4_2","doi-asserted-by":"publisher","DOI":"10.1177\/26339137221114874"},{"key":"e_1_3_4_5_2","volume-title":"Programming Collective Intelligence: Building Smart Web 2.0 Applications","author":"Segaran T.","year":"2007","unstructured":"T. Segaran, Programming Collective Intelligence: Building Smart Web 2.0 Applications (\u201cO\u2019Reilly Media, Inc.\u201d, 2007)."},{"key":"e_1_3_4_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-08681-1"},{"key":"e_1_3_4_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2020.101438"},{"key":"e_1_3_4_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-13-8126-3_4"},{"key":"e_1_3_4_9_2","doi-asserted-by":"publisher","DOI":"10.1080\/23265507.2018.1557072"},{"key":"e_1_3_4_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-0716-2197-4_3"},{"key":"e_1_3_4_11_2","doi-asserted-by":"publisher","DOI":"10.3390\/su11030915"},{"key":"e_1_3_4_12_2","unstructured":"M. Revel D. Halpern A. Berinsky A. Jadbabaie \u201cLiquid democracy in practice: An empirical analysis of its epistemic performance\u201d in EAAMO '22: Equity and Access in Algorithms Mechanisms and Optimization Arlington VA USA (Association for Computing Machinery New York NY United States 2022) (available at: https:\/\/daniel-halpern.com\/files\/liquid-in-practice.pdf)."},{"key":"e_1_3_4_13_2","doi-asserted-by":"crossref","unstructured":"J. Campbell A. Casella L. de Lara V. Mooers D. Ravindran Liquid democracy. Two Experiments on delegation in voting w30794 (National Bureau of Economic Research 2022) 10.3386\/w30794.","DOI":"10.3386\/w30794"},{"key":"e_1_3_4_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2019.2930913"},{"key":"e_1_3_4_15_2","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v12i1.15033"},{"key":"e_1_3_4_16_2","doi-asserted-by":"publisher","DOI":"10.1257\/aer.100.1.541"},{"key":"e_1_3_4_17_2","doi-asserted-by":"publisher","DOI":"10.3233\/IP-170065"},{"key":"e_1_3_4_18_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-23223-8"},{"key":"e_1_3_4_19_2","doi-asserted-by":"publisher","DOI":"10.1177\/14614448211044025"},{"key":"e_1_3_4_20_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1008636108"},{"key":"e_1_3_4_21_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1703695114"},{"key":"e_1_3_4_22_2","volume-title":"Understanding Knowledge as a Commons: From Theory to Practice","author":"Hess C.","year":"2011","unstructured":"C. Hess, E. Ostrom, Understanding Knowledge as a Commons: From Theory to Practice (MIT Press, 2011)."},{"key":"e_1_3_4_23_2","unstructured":"K. R. McKee Social diversity and social preferences in mixed-motive reinforcement learning. arXiv [cs.MA] [Preprint] (2020). http:\/\/arxiv.org\/abs\/2002.02325 (Accessed 12 February 2020)."},{"key":"e_1_3_4_24_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1618722114"},{"key":"e_1_3_4_25_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1407301111"},{"key":"e_1_3_4_26_2","doi-asserted-by":"publisher","DOI":"10.1037\/mot0000116"},{"key":"e_1_3_4_27_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-023-01540-w"},{"key":"e_1_3_4_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366701"},{"key":"e_1_3_4_29_2","first-page":"2786","article-title":"Multi-armed bandits in multi-agent networks","author":"Shahrampour S.","year":"2017","unstructured":"S. Shahrampour, A. Rakhlin, A. Jadbabaie, \u201cMulti-armed bandits in multi-agent networks\u201d in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP, 2017), pp. 2786\u20132790.","journal-title":"2017 IEEE International Conference on Acoustics, Speech and Signal Processing"},{"key":"e_1_3_4_30_2","unstructured":"V. Kuleshov D. Precup Algorithms for multi-armed bandit problems. arXiv [cs.AI] [Preprint] (2014). http:\/\/arxiv.org\/abs\/1402.6028 (Accessed 25 February 2014)."},{"key":"e_1_3_4_31_2","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton R. S.","year":"2018","unstructured":"R. S. Sutton, A. G. Barto, Reinforcement Learning: An Introduction (MIT Press, ed. 2, 2018).","edition":"2"},{"key":"e_1_3_4_32_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-021-22852-3"},{"key":"e_1_3_4_33_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10551-018-4083-z"},{"key":"e_1_3_4_34_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1817392116"},{"key":"e_1_3_4_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jrp.2006.02.001"},{"key":"e_1_3_4_36_2","first-page":"6999","article-title":"The 9-SRA Scale: A Simplified 9-Items Version of the SRA Scale to Assess Altruism","volume":"13","author":"Manzur E.","year":"2021","unstructured":"E. Manzur, S. Olavarrieta, The 9-SRA Scale: A Simplified 9-Items Version of the SRA Scale to Assess Altruism. Sustain. Sci. Pract. Policy. 13, 6999 (2021).","journal-title":"Sustain. Sci. Pract. Policy."},{"key":"e_1_3_4_37_2","doi-asserted-by":"publisher","DOI":"10.2307\/1912352"},{"key":"e_1_3_4_38_2","first-page":"213","article-title":"When extrinsic incentives displace intrinsic motivation: designing legal carrots and sticks to confront the challenge of motivational crowding-out","volume":"33","author":"Underhill K.","year":"2016","unstructured":"K. Underhill, When extrinsic incentives displace intrinsic motivation: designing legal carrots and sticks to confront the challenge of motivational crowding-out. Yale J. Regul. 33, 213 (2016).","journal-title":"Yale J. Regul."},{"key":"e_1_3_4_39_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-022-05512-4"},{"key":"e_1_3_4_40_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i5.20417"},{"key":"e_1_3_4_41_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-022-01383-x"},{"key":"e_1_3_4_42_2","first-page":"10659","article-title":"Gibbs sampling with people","volume":"33","author":"Harrison P.","year":"2020","unstructured":"P. Harrison , Gibbs sampling with people. Adv. Neural Inf. Process. Syst. 33, 10659\u201310671 (2020).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_4_43_2","unstructured":"O. Tchernichovski N. Jacoby S. Frey D. Conley Rating accuracy in collective information. Open Science Framework. https:\/\/osf.io\/d953m\/. Deposited 5 October 2023."}],"container-title":["Proceedings of the National Academy of Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pnas.org\/doi\/pdf\/10.1073\/pnas.2311497120","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,6]],"date-time":"2023-11-06T15:32:41Z","timestamp":1699284761000},"score":1,"resource":{"primary":{"URL":"https:\/\/pnas.org\/doi\/10.1073\/pnas.2311497120"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,6]]},"references-count":43,"journal-issue":{"issue":"46","published-print":{"date-parts":[[2023,11,14]]}},"alternative-id":["10.1073\/pnas.2311497120"],"URL":"https:\/\/doi.org\/10.1073\/pnas.2311497120","relation":{},"ISSN":["0027-8424","1091-6490"],"issn-type":[{"value":"0027-8424","type":"print"},{"value":"1091-6490","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,6]]},"assertion":[{"value":"2023-07-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-05","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2311497120"}}