{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T21:11:48Z","timestamp":1778620308393,"version":"3.51.4"},"reference-count":58,"publisher":"Maximum Academic Press","license":[{"start":{"date-parts":[[2018,4,17]],"date-time":"2018-04-17T00:00:00Z","timestamp":1523923200000},"content-version":"unspecified","delay-in-days":106,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["The Knowledge Engineering Review"],"published-print":{"date-parts":[[2018]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>With the growth of the Internet, crowdsourcing has become a popular way to perform intelligence tasks that hitherto would be either performed internally within an organization or not undertaken due to prohibitive costs and the lack of an appropriate communications infrastructure. In crowdsourcing systems, whereby multiple agents are not under the direct control of a system designer, it cannot be assumed that agents will act in a manner that is consistent with the objectives of the system designer or principal agent. In situations whereby agents\u2019 goals are to maximize their return in crowdsourcing systems that offer financial or other rewards, strategies will be adopted by agents to game the system if appropriate mitigating measures are not put in place. The motivational and incentivization research space is quite large; it incorporates diverse techniques from a variety of different disciplines including behavioural economics, incentive theory, and game theory. This paper specifically focusses on game theoretic approaches to the problem in the crowdsourcing domain and places it in the context of the wider research landscape. It provides a survey of incentive engineering techniques that enable the creation of apt incentive structures in a range of different scenarios.<\/jats:p>","DOI":"10.1017\/s0269888918000061","type":"journal-article","created":{"date-parts":[[2018,4,17]],"date-time":"2018-04-17T04:45:40Z","timestamp":1523940340000},"source":"Crossref","is-referenced-by-count":18,"title":["A survey of incentive engineering for crowdsourcing"],"prefix":"10.48130","volume":"33","author":[{"given":"Conor","family":"Muldoon","sequence":"first","affiliation":[]},{"given":"Michael J.","family":"O\u2019Grady","sequence":"additional","affiliation":[]},{"given":"Gregory M. P.","family":"O\u2019Hare","sequence":"additional","affiliation":[]}],"member":"27968","published-online":{"date-parts":[[2018,4,17]]},"reference":[{"key":"S0269888918000061_ref21","unstructured":"Howe J. 2006a. Crowdsourcing: a definition. Retrieved 3 June 2006 from http:\/\/crowdsourcing.typepad.com\/cs\/2006\/06\/crowdsourcing_a.html."},{"key":"S0269888918000061_ref54","doi-asserted-by":"publisher","DOI":"10.1145\/1386790.1386829"},{"key":"S0269888918000061_ref39","doi-asserted-by":"publisher","DOI":"10.3390\/ijgi5050055"},{"key":"S0269888918000061_ref50","doi-asserted-by":"publisher","DOI":"10.1111\/j.1540-6261.1961.tb02789.x"},{"key":"S0269888918000061_ref3","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9760.2006.00235.x"},{"key":"S0269888918000061_ref20","doi-asserted-by":"publisher","DOI":"10.1016\/j.elerap.2013.01.004"},{"key":"S0269888918000061_ref38","doi-asserted-by":"publisher","DOI":"10.1109\/MGRS.2015.2498840"},{"key":"S0269888918000061_ref55","doi-asserted-by":"publisher","DOI":"10.1109\/INFCOM.2012.6195597"},{"key":"S0269888918000061_ref2000","first-page":"375","article-title":"Intelligent Sensing for Citizen Science - Challenges and Future Directions","year":"2016","journal-title":"Mobile Networks and Applications"},{"key":"S0269888918000061_ref12","unstructured":"Dayama P. , Narayanaswamy B. , Garg D. & Narahari Y. 2015. Truthful interval cover mechanisms for crowdsourcing applications. In Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems, 1091\u20131099. International Foundation for Autonomous Agents and Multiagent Systems."},{"key":"S0269888918000061_ref56","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2016.7509386"},{"key":"S0269888918000061_ref16","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.2013.1832"},{"key":"S0269888918000061_ref36","doi-asserted-by":"publisher","DOI":"10.1016\/S0022-0000(03)00044-8"},{"key":"S0269888918000061_ref42","doi-asserted-by":"publisher","DOI":"10.1145\/1958824.1958865"},{"key":"S0269888918000061_ref31","doi-asserted-by":"publisher","DOI":"10.2307\/1969529"},{"key":"S0269888918000061_ref1","doi-asserted-by":"publisher","DOI":"10.3391\/mbi.2015.6.2.12"},{"key":"S0269888918000061_ref24","doi-asserted-by":"publisher","DOI":"10.1016\/j.cosrev.2009.04.003"},{"key":"S0269888918000061_ref48","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2014.04.005"},{"key":"S0269888918000061_ref11","unstructured":"David G. , Michael R. , Erwin F. & Martin S. 2012. Crowdsourcing information systems: definition, typology and design. In ICIS 2012 The 33rd International Conference on Information Systems, Joey F George (ed.). Association for Information Systems\/AIS Electronic Library (AISeL), 3562\u20133572."},{"key":"S0269888918000061_ref4","first-page":"133","article-title":"As-if behavioral economics: Neoclassical economics in disguise?","volume":"18","author":"Berg","year":"2010","journal-title":"History of Economic Ideas"},{"key":"S0269888918000061_ref7","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/9693.001.0001"},{"key":"S0269888918000061_ref8","doi-asserted-by":"publisher","DOI":"10.1016\/S1364-6613(03)00094-9"},{"key":"S0269888918000061_ref9","doi-asserted-by":"publisher","DOI":"10.1007\/BF03392017"},{"key":"S0269888918000061_ref15","unstructured":"Endriss U. , Kraus S. , Lang J. & Wooldridge M. 2011. Designing incentives for boolean games. In The 10th International Conference on Autonomous Agents and Multiagent Systems-Volume 1, 79\u201386. International Foundation for Autonomous Agents and Multiagent Systems."},{"key":"S0269888918000061_ref43","first-page":"1","article-title":"Double or nothing: multiplicative incentive mechanisms for crowdsourcing","volume":"1","author":"Shah","year":"2015","journal-title":"Advances in Neural Information Processing Systems"},{"key":"S0269888918000061_ref17","unstructured":"Ghezzi A. , Gabelloni D. , Martini A. & Natalicchio A. 2017. Crowdsourcing: a review and suggestions for future research. International Journal of Management Reviews 0, 1\u201321."},{"key":"S0269888918000061_ref18","doi-asserted-by":"publisher","DOI":"10.2307\/1914085"},{"key":"S0269888918000061_ref52","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2016.7509395"},{"key":"S0269888918000061_ref1000","unstructured":"Higgins C. I. , Williams J. , Leibovici D. G. , Simonis I. , Davis M. J. , Muldoon C. , van Genuchten P. , O\u2019Hare G. M. P. & Wiemann S. 2016. Citizen OBservatory WEB (COBWEB): a generic infrastructure platform to facilitate the collection of citizen science data for environmental monitoring. International Journal of Spatial Data Infrastructures Research 11(1), 20\u201348."},{"key":"S0269888918000061_ref30","doi-asserted-by":"crossref","DOI":"10.2307\/j.ctvjsf522","volume-title":"Game Theory. Analysis of Conflict","author":"Myerson","year":"2013"},{"key":"S0269888918000061_ref25","doi-asserted-by":"crossref","unstructured":"Mao A. , Kamar E. , Chen Y. , Horvitz E. , Schwamb M. E. , Lintott C. J. & Smith A. M. 2013. Volunteering versus work for pay: incentives and tradeoffs in crowdsourcing. In First AAAI Conference on Human Computation and Crowdsourcing.","DOI":"10.1609\/hcomp.v1i1.13075"},{"key":"S0269888918000061_ref26","unstructured":"McLeod S.A. 2007. BF Skinner: Operant conditioning. Retrieved September 9, 2009."},{"key":"S0269888918000061_ref28","doi-asserted-by":"publisher","DOI":"10.1002\/joc.4210"},{"key":"S0269888918000061_ref29","doi-asserted-by":"publisher","DOI":"10.1287\/moor.6.1.58"},{"key":"S0269888918000061_ref32","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511800481"},{"key":"S0269888918000061_ref33","doi-asserted-by":"publisher","DOI":"10.1126\/science.aaa8403"},{"key":"S0269888918000061_ref34","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2015.03.007"},{"key":"S0269888918000061_ref35","doi-asserted-by":"publisher","DOI":"10.1016\/S0014-2921(01)00207-0"},{"key":"S0269888918000061_ref37","doi-asserted-by":"publisher","DOI":"10.1145\/1785414.1785439"},{"key":"S0269888918000061_ref40","doi-asserted-by":"publisher","DOI":"10.1016\/j.isprsjprs.2014.06.016"},{"key":"S0269888918000061_ref45","doi-asserted-by":"publisher","DOI":"10.1145\/1378704.1378721"},{"key":"S0269888918000061_ref47","doi-asserted-by":"publisher","DOI":"10.1145\/1837885.1837895"},{"key":"S0269888918000061_ref49","doi-asserted-by":"publisher","DOI":"10.1007\/BF00122574"},{"key":"S0269888918000061_ref51","unstructured":"Von Neumann J. & Morgenstern O. 1944. Theorem 3.1.18, Theory of games and economic behavior, Princeton University Press."},{"key":"S0269888918000061_ref14","unstructured":"Ebden M. , Huynh D. , Moreau L. & Roberts S. 2015. Incentive engineering through subgraph matching with application to task allocation. HAIDM."},{"key":"S0269888918000061_ref23","doi-asserted-by":"publisher","DOI":"10.1257\/000282803322655392"},{"key":"S0269888918000061_ref22","first-page":"1","article-title":"The rise of crowdsourcing","volume":"14","author":"Howe","year":"2006","journal-title":"Wired Magazine"},{"key":"S0269888918000061_ref46","doi-asserted-by":"publisher","DOI":"10.1145\/2488388.2488490"},{"key":"S0269888918000061_ref44","doi-asserted-by":"publisher","DOI":"10.1109\/ICC.2015.7248819"},{"key":"S0269888918000061_ref19","doi-asserted-by":"publisher","DOI":"10.1145\/2794400"},{"key":"S0269888918000061_ref13","first-page":"231","article-title":"Incentive engineering for computational resource management","volume":"2","author":"Drexler","year":"1988","journal-title":"The Ecology of Computation"},{"key":"S0269888918000061_ref53","doi-asserted-by":"publisher","DOI":"10.1145\/2348543.2348567"},{"key":"S0269888918000061_ref5","unstructured":"Blum A. & Monsour Y. 2007. Learning, regret minimization, and equilibria. In Algorithmic Game Theory, Nisan, N., Roughgarden, T., Tardos, E. & Vazirani, V. V. (eds). Cambridge University Press, 79\u2013102."},{"key":"S0269888918000061_ref6","doi-asserted-by":"publisher","DOI":"10.4102\/sajim.v19i1.739"},{"key":"S0269888918000061_ref27","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2016.543"},{"key":"S0269888918000061_ref2","doi-asserted-by":"publisher","DOI":"10.1145\/2559152"},{"key":"S0269888918000061_ref41","doi-asserted-by":"publisher","DOI":"10.1080\/13658816.2016.1189556"},{"key":"S0269888918000061_ref10","first-page":"104","article-title":"Organizing knowledge syntheses: a taxonomy of literature reviews","volume":"1","author":"Cooper","year":"1988","journal-title":"Knowledge in Society"}],"container-title":["The Knowledge Engineering Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0269888918000061","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T14:42:10Z","timestamp":1767624130000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0269888918000061\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018]]},"references-count":58,"alternative-id":["S0269888918000061"],"URL":"https:\/\/doi.org\/10.1017\/s0269888918000061","relation":{},"ISSN":["0269-8889","1469-8005"],"issn-type":[{"value":"0269-8889","type":"print"},{"value":"1469-8005","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018]]},"article-number":"e2"}}