{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T05:21:05Z","timestamp":1774675265022,"version":"3.50.1"},"reference-count":85,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2020,2,6]],"date-time":"2020-02-06T00:00:00Z","timestamp":1580947200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2021,1,31]]},"abstract":"<jats:p>Over the last few years, Collective Intelligence (CI) platforms have become a vital resource for learning, problem solving, decision-making, and predictions. This rising interest in the topic has to led to the development of several models and frameworks available in published literature. Unfortunately, most of these models are built around domain-specific requirements, i.e., they are often based on the intuitions of their domain experts and developers. This has created a gap in our knowledge in the theoretical foundations of CI systems and models, in general. In this article, we attempt to fill this gap by conducting a systematic review of CI models and frameworks, identified from a collection of 9,418 scholarly articles published since 2000. Eventually, we contribute by aggregating the available knowledge from 12 CI models into one novel framework and present a generic model that describes CI systems irrespective of their domains. We add to the previously available CI models by providing a more granular view of how different components of CI systems interact. We evaluate the proposed model by examining it with respect to six popular, ongoing CI initiatives available on the Web.<\/jats:p>","DOI":"10.1145\/3368986","type":"journal-article","created":{"date-parts":[[2020,2,6]],"date-time":"2020-02-06T21:54:04Z","timestamp":1581026044000},"page":"1-36","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":89,"title":["Frameworks for Collective Intelligence"],"prefix":"10.1145","volume":"53","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7180-731X","authenticated-orcid":false,"given":"Shweta","family":"Suran","sequence":"first","affiliation":[{"name":"Tallinn University of Technology, Tallinn, Estonia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7389-5955","authenticated-orcid":false,"given":"Vishwajeet","family":"Pattanaik","sequence":"additional","affiliation":[{"name":"Tallinn University of Technology, Tallinn, Estonia"}]},{"given":"Dirk","family":"Draheim","sequence":"additional","affiliation":[{"name":"Tallinn University of Technology, Tallinn, Estonia"}]}],"member":"320","published-online":{"date-parts":[[2020,2,6]]},"reference":[{"key":"e_1_2_2_1_1","first-page":"847","article-title":"Information cascades in the laboratory","volume":"87","author":"Anderson Lisa R.","year":"1997","journal-title":"The American Economic Review"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0956-5221(97)84644-1"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1108\/13673270610650120"},{"key":"e_1_2_2_4_1","first-page":"2","article-title":"Decisions 2.0: The power of collective intelligence","volume":"50","author":"Bonabeau Eric","year":"2009","journal-title":"MIT Sloan Management Review"},{"key":"e_1_2_2_5_1","first-page":"5","article-title":"Swarm intelligence\u2014A whole new way to think about business","volume":"79","author":"Bonabeau E.","year":"2001","journal-title":"Harvard Business Review"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cogsys.2005.11.007"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1177\/1354856507084420"},{"key":"e_1_2_2_8_1","volume-title":"Communication 8 Society 13, 8","author":"Brabham Daren C.","year":"2010"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1134285.1134500"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.techfore.2007.05.006"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1155\/2015\/368273"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1754-9434.2010.01231.x"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998250"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1080\/002077200406543"},{"key":"e_1_2_2_15_1","volume-title":"MCIS 2012 Proceedings. Mediterranean Conference on Information Systems. https:\/\/aisel.aisnet.org\/mcis2012\/14","author":"de Castro Neto Miguel","year":"2012"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1108\/FS-05-2012-0032"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1924421.1924442"},{"key":"e_1_2_2_18_1","first-page":"2","article-title":"Effect of collective intelligence in organizations","volume":"24","author":"Farias dos Passos Ketry Gorete","year":"2012","journal-title":"Transinforma\u00e7\u00e3o"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2009.08.004"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824136"},{"key":"e_1_2_2_21_1","volume-title":"Advances in Intelligent and Soft Computing.","author":"Georgi Sandro"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1038\/438900a"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1177\/1946756713497331"},{"key":"e_1_2_2_24_1","first-page":"1","article-title":"The CEO\u2019s role in business model reinvention","volume":"89","author":"Govindarajan Vijay","year":"2011","journal-title":"Harvard Business Review"},{"key":"e_1_2_2_25_1","volume-title":"Collective intelligence in organizations: Tools and studies. Computer Supported Cooperative Work (CSCW) 21, 4 (01","author":"Grasso Antonietta","year":"2012"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2009.04.012"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1721654.1721691"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.websem.2007.11.011"},{"key":"e_1_2_2_29_1","first-page":"519","article-title":"The use of knowledge in society","volume":"35","author":"Hayek Friedrich August","year":"1945","journal-title":"The American Economic Review"},{"key":"e_1_2_2_30_1","first-page":"30","article-title":"How diversity can drive innovation","volume":"91","author":"Hewlett Sylvia Ann","year":"2013","journal-title":"Harvard Business Review"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0403723101"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1080\/1097198X.2009.10856494"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.4018\/jdsst.2009010105"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/CTS.2011.5928663"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.paid.2011.01.008"},{"key":"e_1_2_2_36_1","volume-title":"The Origins of Order: Self-organization and Selection in Evolution","author":"Kauffman Stuart A."},{"key":"e_1_2_2_37_1","volume-title":"Technical Report EBSE-2007-01. Keele University.","author":"Kitchenham Barbara A.","year":"2007"},{"key":"e_1_2_2_38_1","unstructured":"Gary A Klein. 2004. The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work. Crown Business.  Gary A Klein. 2004. The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work. Crown Business."},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2014.53"},{"key":"e_1_2_2_40_1","article-title":"hackAIR: Towards raising awareness about air quality in Europe by developing a collective online platform","volume":"7","author":"Evangelos Kosmidis","year":"2018","journal-title":"ISPRS International Journal of Geo-Information"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511846427"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.5555\/1671011.1671035"},{"key":"e_1_2_2_43_1","volume-title":"Collective Intelligence: Mankind\u2019s Emerging World in Cyberspace","author":"Levy Pierre","year":"1997"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2009.08.001"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1177\/0894439315591136"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1008636108"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11518-009-5095-0"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.4304\/jetwi.3.3.217-226"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/1643823.1643848"},{"key":"e_1_2_2_50_1","unstructured":"T. W. Malone and M. S. Bernstein. 2015. Handbook of Collective Intelligence. MIT Press. 2015029234 https:\/\/books.google.ee\/books?id&equals;Px3iCgAAQBAJ.  T. W. Malone and M. S. Bernstein. 2015. Handbook of Collective Intelligence. MIT Press. 2015029234 https:\/\/books.google.ee\/books?id&equals;Px3iCgAAQBAJ."},{"key":"e_1_2_2_51_1","first-page":"15","article-title":"Harnessing collective intelligence to address global climate change. Innovations: Technology, Governance","volume":"2","author":"Malone Thomas W.","year":"2007","journal-title":"Globalization"},{"key":"e_1_2_2_52_1","volume-title":"Dellarocas","author":"Malone Thomas W.","year":"2009"},{"key":"e_1_2_2_53_1","volume-title":"Leadership and the wisdom of crowds: How to tap into the collective intelligence of an organization. Strategy 8 Leadership 44, 1","author":"Matzler Kurt","year":"2016"},{"key":"e_1_2_2_54_1","volume-title":"Internet Science","author":"McCrory Gavin"},{"key":"e_1_2_2_55_1","volume-title":"Taylor","author":"Muchnik Lev","year":"2013"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/WICSA.2015.30"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1080\/17439884.2014.929144"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1080\/01969722.2017.1418254"},{"key":"e_1_2_2_59_1","volume-title":"The Knowledge-Creating Company: How Japanese Companies Create the Dynamics of Innovation","author":"Nonaka Ikujiro"},{"key":"e_1_2_2_60_1","volume-title":"Engineering complex systems. Nature 427, 6973","author":"Ottino Julio M.","year":"2004"},{"key":"e_1_2_2_61_1","volume-title":"The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies","author":"S. E. Page and Princeton University Press.","year":"2007"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1177\/0956247814547172"},{"key":"e_1_2_2_63_1","volume-title":"Peters and Richard Heraud","author":"Michael","year":"2015"},{"key":"e_1_2_2_64_1","volume-title":"Thierry B\u00fccheler, and Rudolf Marcel F\u00fcchslin.","author":"Pfeifer Rolf","year":"2010"},{"key":"e_1_2_2_65_1","first-page":"16","article-title":"WEB 2.0 learning platform: Harnessing collective intelligence","volume":"8","author":"Rogers P. Clint","year":"2007","journal-title":"Turkish Online Journal of Distance Education"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1057\/ejis.2014.7"},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2012.80"},{"key":"e_1_2_2_69_1","volume-title":"Social media and collective intelligence\u2014Ongoing and future research streams. KI - K\u00fcnstliche Intelligenz 27, 1 (1","author":"Schoder Detlef","year":"2013"},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2009.08.006"},{"key":"e_1_2_2_71_1","unstructured":"Toby Segaran. 2007. Programming Collective Intelligence : Building Smart Web 2.0 Applications. O\u2019Reilly Beijing\/Sebastapol CA.  Toby Segaran. 2007. Programming Collective Intelligence : Building Smart Web 2.0 Applications. O\u2019Reilly Beijing\/Sebastapol CA."},{"key":"e_1_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1109\/UKSim.2013.77"},{"key":"e_1_2_2_73_1","volume-title":"International Conference on Cyber Warfare and Security. Academic Conferences International Limited, 313","author":"Skarzauskiene Aelita","year":"2015"},{"key":"e_1_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.1525\/cmr.2012.54.4.44"},{"key":"e_1_2_2_75_1","volume-title":"Retrieved on","author":"Sullivan Tim","year":"2019"},{"key":"e_1_2_2_76_1","volume-title":"Computational Collective Intelligence","author":"Suran Shweta"},{"key":"e_1_2_2_77_1","unstructured":"James Surowiecki. 2005. The Wisdom of Crowds. Anchor.  James Surowiecki. 2005. The Wisdom of Crowds. Anchor."},{"key":"e_1_2_2_78_1","doi-asserted-by":"publisher","DOI":"10.1162\/106454699568700"},{"key":"e_1_2_2_79_1","volume-title":"What Is Web 2.0: Design Patterns and Business Models for the Next Generation of Software. MPRA Paper","author":"Tim O\u2019Reilly"},{"key":"e_1_2_2_80_1","doi-asserted-by":"crossref","volume-title":"System Design Modeling and Metamodeling","author":"van Gigch John P.","DOI":"10.1007\/978-1-4899-0676-2"},{"key":"e_1_2_2_81_1","doi-asserted-by":"publisher","DOI":"10.1145\/1936254.1936285"},{"key":"e_1_2_2_82_1","unstructured":"Jennifer H. Watkins. 2007. Prediction markets as an aggregation mechanism for collective intelligence. Retrieved from https:\/\/escholarship.org\/uc\/item\/8mg0p0zc.  Jennifer H. Watkins. 2007. Prediction markets as an aggregation mechanism for collective intelligence. Retrieved from https:\/\/escholarship.org\/uc\/item\/8mg0p0zc."},{"key":"e_1_2_2_83_1","doi-asserted-by":"publisher","DOI":"10.1108\/03055721211227273"},{"key":"e_1_2_2_84_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.intell.2011.01.004"},{"key":"e_1_2_2_85_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1193147"},{"key":"e_1_2_2_86_1","volume-title":"Managing Knowledge in a World of Networks","author":"Zettsu Koji"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3368986","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3368986","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:53:05Z","timestamp":1750204385000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3368986"}},"subtitle":["A Systematic Literature Review"],"short-title":[],"issued":{"date-parts":[[2020,2,6]]},"references-count":85,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,1,31]]}},"alternative-id":["10.1145\/3368986"],"URL":"https:\/\/doi.org\/10.1145\/3368986","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,6]]},"assertion":[{"value":"2019-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-10-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-02-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}