{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T16:48:10Z","timestamp":1780418890543,"version":"3.54.1"},"reference-count":33,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2016,3,31]],"date-time":"2016-03-31T00:00:00Z","timestamp":1459382400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Nano-Tera.ch as part of the Opensense2 project"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2016,7,14]]},"abstract":"<jats:p>\n            Crowdsourcing is widely proposed as a method to solve a large variety of judgment tasks, such as classifying website content, peer grading in online courses, or collecting real-world data. As the data reported by workers cannot be verified, there is a tendency to report random data without actually solving the task. This can be countered by making the reward for an answer depend on its consistency with answers given by other workers, an approach called\n            <jats:italic>peer consistency<\/jats:italic>\n            . However, it is obvious that the best strategy in such schemes is for all workers to report the same answer without solving the task.\n          <\/jats:p>\n          <jats:p>Dasgupta and Ghosh [2013] show that, in some cases, exerting high effort can be encouraged in the highest-paying equilibrium. In this article, we present a general mechanism that implements this idea and is applicable to most crowdsourcing settings. Furthermore, we experimentally test the novel mechanism, and validate its theoretical properties.<\/jats:p>","DOI":"10.1145\/2856102","type":"journal-article","created":{"date-parts":[[2016,3,31]],"date-time":"2016-03-31T11:55:39Z","timestamp":1459425339000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":61,"title":["Incentives for Effort in Crowdsourcing Using the Peer Truth Serum"],"prefix":"10.1145","volume":"7","author":[{"given":"Goran","family":"Radanovic","sequence":"first","affiliation":[{"name":"Artificial Intelligence Laboratory, EPFL, Lausanne, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Boi","family":"Faltings","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Laboratory, EPFL, Lausanne, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Radu","family":"Jurca","sequence":"additional","affiliation":[{"name":"Google, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2016,3,31]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/985692.985733"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1462-9011(02)00039-4"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2488388.2488417"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2013.150"},{"key":"e_1_2_2_5_1","volume-title":"Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing (HCOMP'14)","author":"Faltings Boi","year":"2014"},{"key":"e_1_2_2_6_1","volume-title":"Workshop on Crowdsourcing and Online Behavioral Experiments, in Conjunction with ACM EC13","author":"Gao Xi Alice","year":"2013"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/2892753.2892963"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1566374.1566413"},{"key":"e_1_2_2_9_1","volume-title":"Proceedings of the Workshop on Crowdsourcing for Search and Data Mining (CSDM) at the 4th ACM International Conference on Web Search and Data Mining (WSDM\u201911)","author":"Harris Christopher","year":"2011"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2470743"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441847"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/1622716.1622723"},{"key":"e_1_2_2_13_1","volume-title":"Workshop on Social Computing and User Generated Content (EC\u201911)","author":"Jurca Radu","year":"2011"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/2343896.2343988"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-92185-1_23"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.1050.0379"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2010.10.007"},{"key":"e_1_2_2_18_1","volume-title":"A bayesian truth serum for subjective data. Science 34, 5695","author":"Prelec Drazen","year":"2004"},{"key":"e_1_2_2_19_1","unstructured":"Drazen Prelec and Sebastian Seung. 2006. An algorithm that finds truth even if most people are wrong. Working paper.  Drazen Prelec and Sebastian Seung. 2006. An algorithm that finds truth even if most people are wrong. Working paper."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.5555\/2891460.2891576"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/2893873.2893993"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.5555\/2772879.2773289"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/2887007.2887148"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1958824.1958865"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2488388.2488490"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.5555\/2999134.2999149"},{"key":"e_1_2_2_27_1","volume-title":"Proceedings of the 3rd Workshop on Social Computing and User Generated Content (SC\u201913)","author":"Waggoner Bo","year":"2013"},{"key":"e_1_2_2_28_1","volume-title":"Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing (HCOMP\u201914)","author":"Waggoner Bo","year":"2014"},{"key":"e_1_2_2_29_1","volume-title":"Proceedings of the 1st AAAI Conference on Human Computation and Crowdsourcing.","author":"Witkowski Jens"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2229012.2229085"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.5555\/2900929.2900940"},{"key":"e_1_2_2_32_1","volume-title":"Proceedings of the 3rd Workshop on Social Computing and User Generated Content (SC'13)","author":"Witkowski Jens"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.5555\/2615731.2615773"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2856102","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2856102","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:54:11Z","timestamp":1750222451000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2856102"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,3,31]]},"references-count":33,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2016,7,14]]}},"alternative-id":["10.1145\/2856102"],"URL":"https:\/\/doi.org\/10.1145\/2856102","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,3,31]]},"assertion":[{"value":"2015-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-03-31","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}