{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T03:08:46Z","timestamp":1771556926979,"version":"3.50.1"},"reference-count":72,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2018,6,27]],"date-time":"2018-06-27T00:00:00Z","timestamp":1530057600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Theory and Practice of Social Machines","award":["EP\/J017728\/1"],"award-info":[{"award-number":["EP\/J017728\/1"]}]},{"name":"Stars4All EUHorizon 2020 Framework Programme","award":["688135"],"award-info":[{"award-number":["688135"]}]},{"name":"Digital Economy programme"},{"name":"Engineering and Physical Sciences Research Council on behalf of UK Research and Innovation","award":["EP\/G036926\/1"],"award-info":[{"award-number":["EP\/G036926\/1"]}]},{"name":"UK Engineering and Physical Sciences Research Council (EPSRC) SOCIAM"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Trans. Soc. Comput."],"published-print":{"date-parts":[[2018,6,30]]},"abstract":"<jats:p>The use of game elements within virtual citizen science is increasingly common, promising to bring increased user activity, motivation, and engagement to large-scale scientific projects. However, there is an ongoing debate about whether or not gamifying systems such as these is actually an effective means by which to increase motivation and engagement in the long term. While gamification itself is receiving a large amount of attention, there has been little beyond individual studies to assess its suitability or success for citizen science; similarly, while frameworks exist for assessing citizen science performance, they tend to lack any appreciation of the effects that game elements might have had. We therefore review the literature to determine what the trends are regarding the performance of particular game elements or characteristics in citizen science, and survey existing projects to assess how popular different game features are. Investigating this phenomenon further, we then present the results of a series of interviews carried out with the EyeWire citizen science project team to understand more about how gamification elements are introduced, monitored, and assessed in a live project. Our findings suggest that projects use a range of game elements with points and leaderboards the most popular, particularly in projects that describe themselves as \u201cgames.\u201d Currently, gamification appears to be effective in citizen science for maintaining engagement with existing communities, but shows limited impact for attracting new players.<\/jats:p>","DOI":"10.1145\/3209960","type":"journal-article","created":{"date-parts":[[2018,6,27]],"date-time":"2018-06-27T12:14:36Z","timestamp":1530101676000},"page":"1-39","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":19,"title":["Is Virtual Citizen Science A Game?"],"prefix":"10.1145","volume":"1","author":[{"given":"Elena","family":"Simperl","sequence":"first","affiliation":[{"name":"University of Southampton, United Kingdom"}]},{"given":"Neal","family":"Reeves","sequence":"additional","affiliation":[{"name":"University of Southampton, United Kingdom"}]},{"given":"Chris","family":"Phethean","sequence":"additional","affiliation":[{"name":"University of Southampton, United Kingdom"}]},{"given":"Todd","family":"Lynes","sequence":"additional","affiliation":[{"name":"University of Southampton, United Kingdom"}]},{"given":"Ramine","family":"Tinati","sequence":"additional","affiliation":[{"name":"University of Southampton"}]}],"member":"320","published-online":{"date-parts":[[2018,6,27]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/581571.581573"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2379636.2379653"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2016.07.039"},{"key":"e_1_2_1_4_1","doi-asserted-by":"crossref","unstructured":"Dr. Ivo Blohm and Prof. Dr. Jan M. Leimeister. 2013. Gamification. Business 8 Information Systems Engineering 5 4 (Aug. 2013) 275--278. https:\/\/search.proquest.com\/docview\/1412812105?accountid=13963  Dr. Ivo Blohm and Prof. Dr. Jan M. Leimeister. 2013. Gamification. Business 8 Information Systems Engineering 5 4 (Aug. 2013) 275--278. https:\/\/search.proquest.com\/docview\/1412812105?accountid=13963","DOI":"10.1007\/s12599-013-0273-5"},{"key":"e_1_2_1_5_1","doi-asserted-by":"crossref","unstructured":"John Bohannon. 2014. Online video game plugs players into remote-controlled biochemistry lab. Science 343 6170 (2014) 475--475. arXiv:http:\/\/science.sciencemag.org\/content\/343\/6170\/475.full.pdf.  John Bohannon. 2014. Online video game plugs players into remote-controlled biochemistry lab. Science 343 6170 (2014) 475--475. arXiv:http:\/\/science.sciencemag.org\/content\/343\/6170\/475.full.pdf.","DOI":"10.1126\/science.343.6170.475"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2583008.2583011"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-015-2897-6"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897370"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3102071.3106358"},{"key":"e_1_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Joe Cox Eun Young Oh Brooke Simmons Chris Lintott Karen Masters Gary Graham Anita Greenhill and Kate Holmes. 2015. How is success defined and measured in online citizen science? A case study of Zooniverse projects.  Joe Cox Eun Young Oh Brooke Simmons Chris Lintott Karen Masters Gary Graham Anita Greenhill and Kate Holmes. 2015. How is success defined and measured in online citizen science? A case study of Zooniverse projects.","DOI":"10.1109\/MCSE.2015.65"},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the IFIP WG","volume":"9","author":"Crowston Kevin","year":"2008"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2013.413"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1177\/1075547015609322"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.125.6.627"},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of Gamification: Using Game Design Elements in Non-gaming Contexts, A Workshop at CHI.","author":"Deterding Sebastian","year":"2011"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2212877.2212883"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2181037.2181040"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1979742.1979575"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-007-9041-8"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2910575"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2557262"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2583008.2583019"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2968120.2987738"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1868914.1868998"},{"key":"e_1_2_1_25_1","first-page":"22","article-title":"How is success defined and measured in online citizen science: A case study of Zooniverse projects","volume":"99","author":"Graham G. G.","year":"2015","journal-title":"Computing in Science and Engineering"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1108\/AJIM-11-2015-0182"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.elerap.2013.01.004"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2014.377"},{"key":"e_1_2_1_29_1","first-page":"1","article-title":"The rise of crowdsourcing","volume":"14","author":"Howe Jeff","year":"2006","journal-title":"Wired Magazine"},{"key":"e_1_2_1_30_1","first-page":"61801","article-title":"Systematic analysis of output agreement games: Effects of gaming environment, social interaction, and feedback","volume":"51","author":"Huang Shih-Wen","year":"2012","journal-title":"Urbana"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2393132.2393137"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702204"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2468356.2468553"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.22323\/2.15030205"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13673-017-0090-9"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0031362"},{"key":"e_1_2_1_37_1","doi-asserted-by":"crossref","unstructured":"Jeffrey Laut Francesco Cappa Oded Nov and Maurizio Porfiri. 2016. Increasing citizen science contribution using a virtual peer. Journal of the Association for Information Science and Technology (2016).  Jeffrey Laut Francesco Cappa Oded Nov and Maurizio Porfiri. 2016. Increasing citizen science contribution using a virtual peer. Journal of the Association for Information Science and Technology (2016).","DOI":"10.1002\/asi.23685"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1177\/1555412014559978"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2966.2008.13689.x"},{"key":"e_1_2_1_40_1","unstructured":"Jane McGonigal. 2011. Reality is Broken: Why Games Make us Better And How They Can Change the World. Penguin.   Jane McGonigal. 2011. Reality is Broken: Why Games Make us Better And How They Can Change the World. Penguin."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2468356.2468559"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2016.543"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2017.04.005"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/1940761.1940771"},{"key":"e_1_2_1_45_1","volume-title":"Proceedings of the 11th European Conference on Games Based Learning (ECGBL\u201917)","author":"Pedersen M. K."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2793107.2810293"},{"key":"e_1_2_1_47_1","doi-asserted-by":"crossref","unstructured":"Catherine Pope Sue Ziebland Nicholas Mays and others. 2000. Analysing qualitative data. BMJ 320 7227 (2000) 114--116.  Catherine Pope Sue Ziebland Nicholas Mays and others. 2000. Analysing qualitative data. BMJ 320 7227 (2000) 114--116.","DOI":"10.1136\/bmj.320.7227.114"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531615"},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of iConference","author":"Prestopnik Nathan","year":"2014"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2016.11.035"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2015.05.051"},{"key":"e_1_2_1_52_1","article-title":"Galaxy zoo: Motivations of citizen scientists","volume":"12","author":"Raddick M. Jordan","year":"2013","journal-title":"Astronomy Education Review"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998302"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2888398"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883075"},{"key":"e_1_2_1_56_1","volume-title":"Proceedings of iConference","author":"Rotman Dana","year":"2014"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145238"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1353\/nar.2005.0013"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/2494188.2494210"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2014.09.006"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3102071.3102077"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2967934.2968083"},{"key":"e_1_2_1_63_1","volume-title":"Proceedings of the 9th International Conference on the Foundations of Digital Games.","author":"Siu Kristin"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2012.67"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/CeDEM.2016.14"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.29379\/jedem.v8i3.440"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145362"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/2908131.2908151"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2016.12.074"},{"key":"e_1_2_1_70_1","volume-title":"Proceedings of the11th International Conference on Web and Social Media. https:\/\/eprints.soton.ac.uk\/406181\/","author":"Tinati Ramine","year":"2017"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702420"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2011.207"}],"container-title":["ACM Transactions on Social Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3209960","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3209960","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:39:19Z","timestamp":1750210759000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3209960"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,6,27]]},"references-count":72,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2018,6,30]]}},"alternative-id":["10.1145\/3209960"],"URL":"https:\/\/doi.org\/10.1145\/3209960","relation":{},"ISSN":["2469-7818","2469-7826"],"issn-type":[{"value":"2469-7818","type":"print"},{"value":"2469-7826","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,6,27]]},"assertion":[{"value":"2017-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-06-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}