{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T15:35:09Z","timestamp":1759937709862,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":33,"publisher":"ACM","license":[{"start":{"date-parts":[[2015,10,17]],"date-time":"2015-10-17T00:00:00Z","timestamp":1445040000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2015,10,17]]},"DOI":"10.1145\/2806416.2806460","type":"proceedings-article","created":{"date-parts":[[2015,10,27]],"date-time":"2015-10-27T13:03:43Z","timestamp":1445951023000},"page":"783-790","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Exploiting Document Content for Efficient Aggregation of Crowdsourcing Votes"],"prefix":"10.1145","author":[{"given":"Martin","family":"Davtyan","sequence":"first","affiliation":[{"name":"ETH Zurich, Zurich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Carsten","family":"Eickhoff","sequence":"additional","affiliation":[{"name":"ETH Zurich, Zurich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thomas","family":"Hofmann","sequence":"additional","affiliation":[{"name":"ETH Zurich, Zurich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2015,10,17]]},"reference":[{"doi-asserted-by":"publisher","key":"e_1_3_2_1_1_1","DOI":"10.1145\/1480506.1480508"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_2_1","DOI":"10.1145\/2009916.2010039"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_3_1","DOI":"10.1080\/13691181003624090"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_4_1","DOI":"10.1145\/1008992.1009000"},{"key":"e_1_3_2_1_5_1","volume-title":"Clueweb09 data set","author":"Callan Jamie","year":"2009","unstructured":"Jamie Callan , Mark Hoy , Changkuk Yoo , and Le Zhao . Clueweb09 data set , 2009 . Jamie Callan, Mark Hoy, Changkuk Yoo, and Le Zhao. Clueweb09 data set, 2009."},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_6_1","DOI":"10.1145\/1148170.1148219"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_7_1","DOI":"10.1145\/1924475.1924481"},{"key":"e_1_3_2_1_8_1","first-page":"20","volume-title":"Maximum likelihood estimation of observer error-rates using the em algorithm. Applied statistics","author":"Dawid Alexander Philip","year":"1979","unstructured":"Alexander Philip Dawid and Allan M Skene . Maximum likelihood estimation of observer error-rates using the em algorithm. Applied statistics , pages 20 -- 28 , 1979 . Alexander Philip Dawid and Allan M Skene. Maximum likelihood estimation of observer error-rates using the em algorithm. Applied statistics, pages 20--28, 1979."},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_9_1","DOI":"10.1145\/2488388.2488421"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_10_1","DOI":"10.1007\/s10791-011-9181-9"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_11_1","DOI":"10.1145\/2348283.2348400"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_12_1","DOI":"10.5555\/1866696.1866723"},{"key":"e_1_3_2_1_13_1","first-page":"4","article-title":"Cheat-detection mechanisms for crowdsourcing. University of W\u00fcrzburg","author":"Hirth Matthias","year":"2010","unstructured":"Matthias Hirth , Tobias Ho\u00dffeld , and Phuoc Tran-Gia . Cheat-detection mechanisms for crowdsourcing. University of W\u00fcrzburg , Tech. Rep , 4 , 2010 . Matthias Hirth, Tobias Ho\u00dffeld, and Phuoc Tran-Gia. Cheat-detection mechanisms for crowdsourcing. University of W\u00fcrzburg, Tech. Rep, 4, 2010.","journal-title":"Tech. Rep"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_14_1","DOI":"10.1007\/978-3-642-28997-2_16"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_15_1","DOI":"10.1145\/1982624.1982625"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_16_1","DOI":"10.1145\/775047.775067"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_17_1","DOI":"10.1287\/opre.2013.1235"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_18_1","DOI":"10.1145\/2063576.2063860"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_19_1","DOI":"10.1007\/s10791-012-9205-0"},{"key":"e_1_3_2_1_20_1","first-page":"21","volume-title":"SIGIR 2009 Workshop on the Future of IR Evaluation","author":"Kazai Gabriella","year":"2009","unstructured":"Gabriella Kazai and Natasa Milic-Frayling . On the evaluation of the quality of relevance assessments collected through crowdsourcing . In SIGIR 2009 Workshop on the Future of IR Evaluation , page 21 , 2009 . Gabriella Kazai and Natasa Milic-Frayling. On the evaluation of the quality of relevance assessments collected through crowdsourcing. In SIGIR 2009 Workshop on the Future of IR Evaluation, page 21, 2009."},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_21_1","DOI":"10.1145\/1357054.1357127"},{"key":"e_1_3_2_1_22_1","volume-title":"Proceedings of the text retrieval conference (TREC)","author":"Lease Matthew","year":"2011","unstructured":"Matthew Lease and Gabriella Kazai . Overview of the trec 2011 crowdsourcing track . In Proceedings of the text retrieval conference (TREC) , 2011 . Matthew Lease and Gabriella Kazai. Overview of the trec 2011 crowdsourcing track. In Proceedings of the text retrieval conference (TREC), 2011."},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_23_1","DOI":"10.1145\/2093346.2093356"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_24_1","DOI":"10.1145\/1998076.1998108"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_25_1","DOI":"10.5555\/2074022.2074067"},{"key":"e_1_3_2_1_26_1","volume-title":"SIGIR 2011 workshop on crowdsourcing for information retrieval (CIR)","author":"Tang Wei","year":"2011","unstructured":"Wei Tang and Matthew Lease . Semi-supervised consensus labeling for crowdsourcing . In SIGIR 2011 workshop on crowdsourcing for information retrieval (CIR) , 2011 . Wei Tang and Matthew Lease. Semi-supervised consensus labeling for crowdsourcing. In SIGIR 2011 workshop on crowdsourcing for information retrieval (CIR), 2011."},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_27_1","DOI":"10.1108\/eb026557"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_28_1","DOI":"10.5555\/648264.753539"},{"key":"e_1_3_2_1_29_1","volume-title":"et al. TREC: Experiment and evaluation in information retrieval","author":"Voorhees Ellen M","year":"2005","unstructured":"Ellen M Voorhees , Donna K Harman , et al. TREC: Experiment and evaluation in information retrieval , volume 1 . MIT press Cambridge , 2005 . Ellen M Voorhees, Donna K Harman, et al. TREC: Experiment and evaluation in information retrieval, volume 1. MIT press Cambridge, 2005."},{"key":"e_1_3_2_1_30_1","first-page":"21","volume-title":"Proc. ACM SIGIR Workshop on Crowdsourcing for Information Retrieval (CIR'11)","author":"Vuurens Jeroen","year":"2011","unstructured":"Jeroen Vuurens , Arjen P de Vries , and Carsten Eickhoff . How much spam can you take? an analysis of crowdsourcing results to increase accuracy . In Proc. ACM SIGIR Workshop on Crowdsourcing for Information Retrieval (CIR'11) , pages 21 -- 26 , 2011 . Jeroen Vuurens, Arjen P de Vries, and Carsten Eickhoff. How much spam can you take? an analysis of crowdsourcing results to increase accuracy. In Proc. ACM SIGIR Workshop on Crowdsourcing for Information Retrieval (CIR'11), pages 21--26, 2011."},{"key":"e_1_3_2_1_31_1","volume-title":"Gaussian processes for machine learning. the MIT Press, 2(3):4","author":"Williams Christopher KI","year":"2006","unstructured":"Christopher KI Williams and Carl Edward Rasmussen . Gaussian processes for machine learning. the MIT Press, 2(3):4 , 2006 . Christopher KI Williams and Carl Edward Rasmussen. Gaussian processes for machine learning. the MIT Press, 2(3):4, 2006."},{"key":"e_1_3_2_1_32_1","first-page":"1161","volume-title":"Proceedings of the 28th international conference on machine learning (ICML-11)","author":"Yan Yan","year":"2011","unstructured":"Yan Yan , Glenn M Fung , R\u00f3mer Rosales , and Jennifer G Dy . Active learning from crowds . In Proceedings of the 28th international conference on machine learning (ICML-11) , pages 1161 -- 1168 , 2011 . Yan Yan, Glenn M Fung, R\u00f3mer Rosales, and Jennifer G Dy. Active learning from crowds. In Proceedings of the 28th international conference on machine learning (ICML-11), pages 1161--1168, 2011."},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_33_1","DOI":"10.1109\/INFCOM.2012.6195597"}],"event":{"sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGIR ACM Special Interest Group on Information Retrieval"],"acronym":"CIKM'15","name":"CIKM'15: 24th ACM International Conference on Information and Knowledge Management","location":"Melbourne Australia"},"container-title":["Proceedings of the 24th ACM International on Conference on Information and Knowledge Management"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2806416.2806460","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2806416.2806460","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T05:43:10Z","timestamp":1750225390000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2806416.2806460"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,10,17]]},"references-count":33,"alternative-id":["10.1145\/2806416.2806460","10.1145\/2806416"],"URL":"https:\/\/doi.org\/10.1145\/2806416.2806460","relation":{},"subject":[],"published":{"date-parts":[[2015,10,17]]},"assertion":[{"value":"2015-10-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}