{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T03:47:11Z","timestamp":1772164031951,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":28,"publisher":"ACM","license":[{"start":{"date-parts":[[2013,6,17]],"date-time":"2013-06-17T00:00:00Z","timestamp":1371427200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2013,6,17]]},"DOI":"10.1145\/2465529.2465761","type":"proceedings-article","created":{"date-parts":[[2013,6,18]],"date-time":"2013-06-18T08:36:08Z","timestamp":1371544568000},"page":"81-92","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":50,"title":["Efficient crowdsourcing for multi-class labeling"],"prefix":"10.1145","author":[{"given":"David R.","family":"Karger","sequence":"first","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sewoong","family":"Oh","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Devavrat","family":"Shah","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2013,6,17]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"Casting Words. http:\/\/castingwords.com.  Casting Words. http:\/\/castingwords.com."},{"key":"e_1_3_2_1_2_1","unstructured":"Crowd Flower. http:\/\/crowd ower.com.  Crowd Flower. http:\/\/crowd ower.com."},{"key":"e_1_3_2_1_3_1","unstructured":"Crowd Spring. http:\/\/www.crowdspring.com.  Crowd Spring. http:\/\/www.crowdspring.com."},{"key":"e_1_3_2_1_4_1","unstructured":"ESP game. http:\/\/www.espgame.org.  ESP game. http:\/\/www.espgame.org."},{"key":"e_1_3_2_1_5_1","unstructured":"Soylent. http:\/\/projects.csail.mit.edu\/soylent\/.  Soylent. http:\/\/projects.csail.mit.edu\/soylent\/."},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047201"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866078"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511814068","volume-title":"Random Graphs","author":"Bollob\u00e1s B.","year":"2001","unstructured":"B. Bollob\u00e1s . Random Graphs . Cambridge University Press , Jan. 2001 . B. Bollob\u00e1s. Random Graphs. Cambridge University Press, Jan. 2001."},{"key":"e_1_3_2_1_9_1","series-title":"Series C (Applied Statistics), 28(1):20--28","volume-title":"Maximum likelihood estimation of observer error-rates using the em algorithm. Journal of the Royal Statistical Society","author":"Dawid A. P.","year":"1979","unstructured":"A. P. Dawid and A. M. Skene . Maximum likelihood estimation of observer error-rates using the em algorithm. Journal of the Royal Statistical Society . Series C (Applied Statistics), 28(1):20--28 , 1979 . A. P. Dawid and A. M. Skene. Maximum likelihood estimation of observer error-rates using the em algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics), 28(1):20--28, 1979."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1977.tb01600.x"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1002\/rsa.v27:2"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/73007.73063"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1993574.1993599"},{"key":"e_1_3_2_1_14_1","volume-title":"Crowd counting a crowd","author":"Holmes S.","year":"2011","unstructured":"S. Holmes . Crowd counting a crowd . March 2011 , Statistics Seminar, Stanford University . S. Holmes. Crowd counting a crowd. March 2011, Statistics Seminar, Stanford University."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837885.1837906"},{"key":"e_1_3_2_1_16_1","first-page":"921","volume-title":"Advances in neural information processing systems","author":"Jin R.","year":"2003","unstructured":"R. Jin and Z. Ghahramani . Learning with multiple labels . In Advances in neural information processing systems , pages 921 -- 928 , 2003 . R. Jin and Z. Ghahramani. Learning with multiple labels. In Advances in neural information processing systems, pages 921--928, 2003."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/Allerton.2011.6120180"},{"key":"e_1_3_2_1_18_1","volume-title":"Budget-optimal task allocation for reliable crowd sourcing systems","author":"Karger D. R.","year":"2011","unstructured":"D. R. Karger , S. Oh , and D. Shah . Budget-optimal task allocation for reliable crowd sourcing systems . 2011 . http:\/\/arxiv.org\/abs\/1110.3564. D. R. Karger, S. Oh, and D. Shah. Budget-optimal task allocation for reliable crowd sourcing systems. 2011. http:\/\/arxiv.org\/abs\/1110.3564."},{"key":"e_1_3_2_1_19_1","first-page":"1953","volume-title":"Advances in neural information processing systems","author":"Karger D. R.","year":"2011","unstructured":"D. R. Karger , S. Oh , and D. Shah . Iterative learning for reliable crowdsourcing systems . In Advances in neural information processing systems , pages 1953 -- 1961 , 2011 . D. R. Karger, S. Oh, and D. Shah. Iterative learning for reliable crowdsourcing systems. In Advances in neural information processing systems, pages 1953--1961, 2011."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2010.2046205"},{"key":"e_1_3_2_1_21_1","first-page":"701","volume-title":"Advances in Neural Information Processing Systems 25","author":"Liu Q.","year":"2012","unstructured":"Q. Liu , J. Peng , and A. Ihler . Variational inference for crowdsourcing . In Advances in Neural Information Processing Systems 25 , pages 701 -- 709 , 2012 . Q. Liu, J. Peng, and A. Ihler. Variational inference for crowdsourcing. In Advances in Neural Information Processing Systems 25, pages 701--709, 2012."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.5555\/2503308.2188401"},{"key":"e_1_3_2_1_23_1","first-page":"1297","article-title":"Learning from crowds","volume":"99","author":"Raykar V. C.","year":"2010","unstructured":"V. C. Raykar , S. Yu , L. H. Zhao , G. H. Valadez , C. Florin , L. Bogoni , and L. Moy . Learning from crowds . J. Mach. Learn. Res. , 99 : 1297 -- 1322 , August 2010 . V. C. Raykar, S. Yu, L. H. Zhao, G. H. Valadez, C. Florin, L. Bogoni, and L. Moy. Learning from crowds. J. Mach. Learn. Res., 99:1297--1322, August 2010.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511791338","volume-title":"Modern Coding Theory","author":"Richardson T.","year":"2008","unstructured":"T. Richardson and R. Urbanke . Modern Coding Theory . Cambridge University Press , march 2008 . T. Richardson and R. Urbanke. Modern Coding Theory. Cambridge University Press, march 2008."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02579208"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1401890.1401965"},{"key":"e_1_3_2_1_27_1","first-page":"2424","volume-title":"Advances in Neural Information Processing Systems","author":"Welinder P.","year":"2010","unstructured":"P. Welinder , S. Branson , S. Belongie , and P. Perona . The multidimensional wisdom of crowds . In Advances in Neural Information Processing Systems , pages 2424 -- 2432 , 2010 . P. Welinder, S. Branson, S. Belongie, and P. Perona. The multidimensional wisdom of crowds. In Advances in Neural Information Processing Systems, pages 2424--2432, 2010."},{"key":"e_1_3_2_1_28_1","first-page":"2035","volume-title":"Advances in Neural Information Processing Systems","volume":"22","author":"Whitehill J.","year":"2009","unstructured":"J. Whitehill , P. Ruvolo , T. Wu , J. Bergsma , and J. Movellan . Whose vote should count more: Optimal integration of labels from labelers of unknown expertise . In Advances in Neural Information Processing Systems , volume 22 , pages 2035 -- 2043 , 2009 . J. Whitehill, P. Ruvolo, T. Wu, J. Bergsma, and J. Movellan. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In Advances in Neural Information Processing Systems, volume 22, pages 2035--2043, 2009."}],"event":{"name":"SIGMETRICS '13: ACM SIGMETRICS \/ International Conference on Measurement and Modeling of Computer Systems","location":"Pittsburgh PA USA","acronym":"SIGMETRICS '13","sponsor":["SIGMETRICS ACM Special Interest Group on Measurement and Evaluation"]},"container-title":["Proceedings of the ACM SIGMETRICS\/international conference on Measurement and modeling of computer systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2465529.2465761","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2465529.2465761","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:39:16Z","timestamp":1750221556000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2465529.2465761"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,6,17]]},"references-count":28,"alternative-id":["10.1145\/2465529.2465761","10.1145\/2465529"],"URL":"https:\/\/doi.org\/10.1145\/2465529.2465761","relation":{"is-identical-to":[{"id-type":"doi","id":"10.1145\/2494232.2465761","asserted-by":"object"}]},"subject":[],"published":{"date-parts":[[2013,6,17]]},"assertion":[{"value":"2013-06-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}