{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,19]],"date-time":"2026-01-19T14:55:29Z","timestamp":1768834529302,"version":"3.49.0"},"publisher-location":"New York, New York, USA","reference-count":35,"publisher":"ACM Press","license":[{"start":{"date-parts":[[2018,1,1]],"date-time":"2018-01-01T00:00:00Z","timestamp":1514764800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018]]},"DOI":"10.1145\/3178876.3186147","type":"proceedings-article","created":{"date-parts":[[2018,4,13]],"date-time":"2018-04-13T15:53:48Z","timestamp":1523634828000},"page":"659-668","source":"Crossref","is-referenced-by-count":16,"title":["On the Causal Effect of Badges"],"prefix":"10.1145","author":[{"given":"Tomasz","family":"Kusmierczyk","sequence":"first","affiliation":[{"name":"Norwegian University of Science and Technology, Trondheim, Norway"}]},{"given":"Manuel","family":"Gomez-Rodriguez","sequence":"additional","affiliation":[{"name":"Max Planck Institute for Software Systems, Kaiserslautern, Germany"}]}],"member":"320","reference":[{"key":"key-10.1145\/3178876.3186147-1","doi-asserted-by":"crossref","unstructured":"Odd Aalen, Ornulf Borgan, and Hakon Gjessing . 2008. Survival and event history analysis: a process point of view. Springer Science & Business Media.","DOI":"10.1007\/978-0-387-68560-1"},{"key":"key-10.1145\/3178876.3186147-2","doi-asserted-by":"crossref","unstructured":"Samuel Abramovich, Christian Schunn, and Ross Mitsuo Higashi . 2013. Are badges useful in education?: It depends upon the type of badge and expertise of learner. Educational Technology Research and Development, Vol. 61, 2 (2013), 217--232.","DOI":"10.1007\/s11423-013-9289-2"},{"key":"key-10.1145\/3178876.3186147-3","doi-asserted-by":"crossref","unstructured":"Tim Althoff, Pranav Jindal, and Jure Leskovec . 2017. Online Actions with Offline Impact: How Online Social Networks Influence Online and Offline User Behavior. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining (WSDM 2017). 537--546.","DOI":"10.1145\/3018661.3018672"},{"key":"key-10.1145\/3178876.3186147-4","doi-asserted-by":"crossref","unstructured":"Ashton Anderson, Daniel P. Huttenlocher, Jon M. Kleinberg, and Jure Leskovec . 2013. Steering user behavior with badges. In Proceedings of the 22nd International World Wide Web Conference (WWW '13). 95--106.","DOI":"10.1145\/2488388.2488398"},{"key":"key-10.1145\/3178876.3186147-5","doi-asserted-by":"crossref","unstructured":"Ashton Anderson, Daniel P. Huttenlocher, Jon M. Kleinberg, and Jure Leskovec . 2014. Engaging with massive online courses. In Proceedings of the 23rd International World Wide Web Conference (WWW '14). 687--698.","DOI":"10.1145\/2566486.2568042"},{"key":"key-10.1145\/3178876.3186147-6","doi-asserted-by":"crossref","unstructured":"Sinan Aral and Dylan Walker . 2012. Identifying influential and susceptible members of social networks. Science, Vol. 337, 6092 (2012), 337--341.","DOI":"10.1126\/science.1215842"},{"key":"key-10.1145\/3178876.3186147-7","doi-asserted-by":"crossref","unstructured":"Benny Bornfeld and Sheizaf Rafaeli . 2017. Gamifying with badges: A big data natural experiment on Stack Exchange. First Monday, Vol. 22, 6 (2017).","DOI":"10.5210\/fm.v22i6.7299"},{"key":"key-10.1145\/3178876.3186147-8","doi-asserted-by":"crossref","unstructured":"Yubo Chen, Qi Wang, and Jinhong Xie . 2011. Online social interactions: A natural experiment on word of mouth versus observational learning. Journal of marketing research Vol. 48, 2 (2011), 238--254.","DOI":"10.1509\/jmkr.48.2.238"},{"key":"key-10.1145\/3178876.3186147-9","doi-asserted-by":"crossref","unstructured":"Rajeev H. Dehejia and Sadek Wahba . 2002. Propensity score-matching methods for nonexperimental causal studies. Review of Economics and statistics Vol. 84, 1 (2002), 151--161.","DOI":"10.1162\/003465302317331982"},{"key":"key-10.1145\/3178876.3186147-10","doi-asserted-by":"crossref","unstructured":"Sebastian Deterding, Miguel Sicart, Lennart Nacke, Kenton O'Hara, and Dan Dixon . 2011. Gamification. using game-design elements in non-gaming contexts Proceedings of the International Conference on Human Factors in Computing Systems (CHI 2011). 2425--2428.","DOI":"10.1145\/1979742.1979575"},{"key":"key-10.1145\/3178876.3186147-11","doi-asserted-by":"crossref","unstructured":"Stephen G. Donald and Kevin Lang . 2007. Inference with difference-in-differences and other panel data. The review of Economics and Statistics Vol. 89, 2 (2007), 221--233.","DOI":"10.1162\/rest.89.2.221"},{"key":"key-10.1145\/3178876.3186147-12","unstructured":"David Easley and Arpita Ghosh . 2016. Incentives, Gamification, and Game Theory: An Economic Approach to Badge Design. ACM Trans. Economics and Comput. Vol. 4, 3 (2016), 16:1--16:26."},{"key":"key-10.1145\/3178876.3186147-13","unstructured":"Oriol Borrás Gené, Margarita Mart'ınez-Nú nez, and Ángel Fidalgo Blanco . 2014. Gamification in MOOC: challenges, opportunities and proposals for advancing MOOC model Proceedings of the Second International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM '14). 215--220."},{"key":"key-10.1145\/3178876.3186147-14","doi-asserted-by":"crossref","unstructured":"Stefan Göbel, Sandro Hardy, Viktor Wendel, Florian Mehm, and Ralf Steinmetz . 2010. Serious games for health: personalized exergames. Proceedings of the 18th International Conference on Multimedia 2010 (MM '10). 1663--1666.","DOI":"10.1145\/1873951.1874316"},{"key":"key-10.1145\/3178876.3186147-15","doi-asserted-by":"crossref","unstructured":"Juho Hamari, David J. Shernoff, Elizabeth Rowe, Brianno Coller, Jodi Asbell-Clarke, and Teon Edwards . 2016. Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior Vol. 54 (2016), 170--179.","DOI":"10.1016\/j.chb.2015.07.045"},{"key":"key-10.1145\/3178876.3186147-16","doi-asserted-by":"crossref","unstructured":"Michael D. Hanus and Jesse Fox . 2015. Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers & Education Vol. 80 (2015), 152--161.","DOI":"10.1016\/j.compedu.2014.08.019"},{"key":"key-10.1145\/3178876.3186147-17","unstructured":"Philipp Herzig, Susanne Strahringer, and Michael Ameling . 2012. Gamification of ERP systems - Exploring gamification effects on user acceptance constructs. In Multikonferenz Wirtschaftsinformatik. 793--804."},{"key":"key-10.1145\/3178876.3186147-18","unstructured":"Robert V. Hogg and Allen T. Craig . 1995. Introduction to mathematical statistics. Prentice Hall."},{"key":"key-10.1145\/3178876.3186147-19","doi-asserted-by":"crossref","unstructured":"Nicole Immorlica, Gregory Stoddard, and Vasilis Syrgkanis . 2015. Social Status and Badge Design. In Proceedings of the 24th International Conference on World Wide Web (WWW '15). 473--483.","DOI":"10.1145\/2736277.2741664"},{"key":"key-10.1145\/3178876.3186147-20","doi-asserted-by":"crossref","unstructured":"Abigail Z. Jacobs, Samuel F. Way, Johan Ugander, and Aaron Clauset . 2015. Assembling thefacebook: Using Heterogeneity to Understand Online Social Network Assembly Proceedings of the 7th ACM Web Science Conference (WebSci '15). 18:1--18:10.","DOI":"10.1145\/2786451.2786477"},{"key":"key-10.1145\/3178876.3186147-21","unstructured":"Daniel Johnson, Sebastian Deterding, Kerri-Ann Kuhn, Aleksandra Staneva, Stoyan Stoyanov, and Leanne Hides . 2016. Gamification for health and wellbeing: A systematic review of the literature. Internet interventions Vol. 6 (2016), 89--106."},{"key":"key-10.1145\/3178876.3186147-22","doi-asserted-by":"crossref","unstructured":"Adam D. I. Kramer, Jamie E. Guillory, and Jeffrey T. Hancock . 2014. Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, Vol. 111, 24 (2014), 8788--8790.","DOI":"10.1073\/pnas.1320040111"},{"key":"key-10.1145\/3178876.3186147-23","unstructured":"Michael Lechner . 2011. The estimation of causal effects by difference-in-difference methods. Foundations and Trends® in Econometrics, Vol. 4, 3 (2011), 165--224."},{"key":"key-10.1145\/3178876.3186147-24","doi-asserted-by":"crossref","unstructured":"Simon McCallum and Costas Boletsis . 2013. Dementia Games: A Literature Review of Dementia-Related Serious Games Proceedings of the 4th International Conference on Serious Games Development and Applications (SGDA 2013). 15--27.","DOI":"10.1007\/978-3-642-40790-1_2"},{"key":"key-10.1145\/3178876.3186147-25","doi-asserted-by":"crossref","unstructured":"Breed D. Meyer . 1995. Natural and quasi-experiments in economics. Journal of business & economic statistics Vol. 13, 2 (1995), 151--161.","DOI":"10.3386\/t0170"},{"key":"key-10.1145\/3178876.3186147-26","doi-asserted-by":"crossref","unstructured":"Benedikt Morschheuser, Juho Hamari, and Jonna Koivisto . 2016. Gamification in Crowdsourcing: A Review. In Proceedings of the 49th Hawaii International Conference on System Sciences (HICSS 2016). 4375--4384.","DOI":"10.1109\/HICSS.2016.543"},{"key":"key-10.1145\/3178876.3186147-27","unstructured":"Cristina Muntean . 2011. Raising engagement in e-learning through gamification Proceedings 6th International Conference on Virtual Learning (ICVL 2011)."},{"key":"key-10.1145\/3178876.3186147-28","unstructured":"Tobias Mutter and Dennis Kundisch . 2014. Behavioral Mechanisms Prompted by Badges: The Goal-Gradient Hypothesis Proceedings of the International Conference on Information Systems - Building a Better World through Information Systems (ICIS 2014)."},{"key":"key-10.1145\/3178876.3186147-29","doi-asserted-by":"crossref","unstructured":"Hüseyin Oktay, Brian J. Taylor, and David D. Jensen . 2010. Causal discovery in social media using quasi-experimental designs Proceedings of the 3rd Workshop on Social Network Mining and Analysis (SNAKDD 2009). 1--9.","DOI":"10.1145\/1964858.1964859"},{"key":"key-10.1145\/3178876.3186147-30","doi-asserted-by":"crossref","unstructured":"Tuan Q. Phan and Edoardo M. Airoldi . 2015. A natural experiment of social network formation and dynamics. Proceedings of the National Academy of Sciences, Vol. 112, 21 (2015), 6595--6600.","DOI":"10.1073\/pnas.1404770112"},{"key":"key-10.1145\/3178876.3186147-31","doi-asserted-by":"crossref","unstructured":"Mark R. Rosenzweig and Kenneth I. Wolpin . 2000. Natural \"natural experiments\" in economics. Journal of Economic Literature Vol. 38, 4 (2000), 827--874.","DOI":"10.1257\/jel.38.4.827"},{"key":"key-10.1145\/3178876.3186147-32","doi-asserted-by":"crossref","unstructured":"Katie Seaborn and Deborah I. Fels . 2015. Gamification in theory and action: A survey. Int. J. Hum.-Comput. Stud. Vol. 74 (2015), 14--31.","DOI":"10.1016\/j.ijhcs.2014.09.006"},{"key":"key-10.1145\/3178876.3186147-33","unstructured":"Elizabeth A. Stuart . 2010. Matching methods for causal inference: A review and a look forward. Statistical science: a review journal of the Institute of Mathematical Statistics, Vol. 25, 1 (2010), 1--21."},{"key":"key-10.1145\/3178876.3186147-34","doi-asserted-by":"crossref","unstructured":"Samuel S. Wilks . 1938. The large-sample distribution of the likelihood ratio for testing composite hypotheses. The Annals of Mathematical Statistics Vol. 9, 1 (1938), 60--62.","DOI":"10.1214\/aoms\/1177732360"},{"key":"key-10.1145\/3178876.3186147-35","doi-asserted-by":"crossref","unstructured":"Jiawei Zhang, Xiangnan Kong, and Philip S. Yu . 2016. Badge System Analysis and Design. In Proceedings of the 2016 IEEE\/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2016).","DOI":"10.1109\/ASONAM.2016.7752274"}],"event":{"name":"the 2018 World Wide Web Conference","location":"Lyon, France","acronym":"WWW '18","number":"2018","sponsor":["SIGWEB, ACM Special Interest Group on Hypertext, Hypermedia, and Web","IW3C2, International World Wide Web Conference Committee"],"start":{"date-parts":[[2018,4,23]]},"end":{"date-parts":[[2018,4,27]]}},"container-title":["Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3178876.3186147","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/dl.acm.org\/ft_gateway.cfm?id=3186147&ftid=1957526&dwn=1","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,3]],"date-time":"2025-07-03T17:25:57Z","timestamp":1751563557000},"score":1,"resource":{"primary":{"URL":"http:\/\/dl.acm.org\/citation.cfm?doid=3178876.3186147"}},"subtitle":[],"proceedings-subject":"World Wide Web","short-title":[],"issued":{"date-parts":[[2018]]},"references-count":35,"URL":"https:\/\/doi.org\/10.1145\/3178876.3186147","relation":{},"subject":[],"published":{"date-parts":[[2018]]}}}