{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:17:12Z","timestamp":1750306632253,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":37,"publisher":"ACM","license":[{"start":{"date-parts":[[2014,6,2]],"date-time":"2014-06-02T00:00:00Z","timestamp":1401667200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2014,6,2]]},"DOI":"10.1145\/2611040.2611047","type":"proceedings-article","created":{"date-parts":[[2014,5,27]],"date-time":"2014-05-27T12:57:10Z","timestamp":1401195430000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["When in Doubt Ask the Crowd"],"prefix":"10.1145","author":[{"given":"Mihai","family":"Georgescu","sequence":"first","affiliation":[{"name":"L3S Research Center, Leibniz Universit\u00e4t Hannover, Appelstr. 9a, 30167 Hannover, Germany"}]},{"given":"Dang Duc","family":"Pham","sequence":"additional","affiliation":[{"name":"L3S Research Center, Leibniz Universit\u00e4t Hannover, Appelstr. 9a, 30167 Hannover, Germany"}]},{"given":"Claudiu S.","family":"Firan","sequence":"additional","affiliation":[{"name":"L3S Research Center, Leibniz Universit\u00e4t Hannover, Appelstr. 9a, 30167 Hannover, Germany"}]},{"given":"Ujwal","family":"Gadiraju","sequence":"additional","affiliation":[{"name":"L3S Research Center, Leibniz Universit\u00e4t Hannover, Appelstr. 9a, 30167 Hannover, Germany"}]},{"given":"Wolfgang","family":"Nejdl","sequence":"additional","affiliation":[{"name":"L3S Research Center, Leibniz Universit\u00e4t Hannover, Appelstr. 9a, 30167 Hannover, Germany"}]}],"member":"320","published-online":{"date-parts":[[2014,6,2]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.5555\/2024236.2024249"},{"key":"e_1_3_2_1_2_1","volume-title":"ILREC","volume":"11","author":"Ambati V.","year":"2010","unstructured":"V. Ambati , S. Vogel , and J. G. Carbonell . Active learning and crowd-sourcing for machine translation . In ILREC , volume 11 . Citeseer , 2010 . V. Ambati, S. Vogel, and J. G. Carbonell. Active learning and crowd-sourcing for machine translation. In ILREC, volume 11. Citeseer, 2010."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835804.1835859"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-008-0098-x"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2003.1234765"},{"key":"e_1_3_2_1_6_1","volume-title":"9th International Conference on Computer Vision Theory and Applications, VISAPP","author":"Chatzilari E.","year":"2014","unstructured":"E. Chatzilari , S. Nikolopoulos , Y. Kompatsiaris , and J. Kittler . Active learning in social context for image classification . In 9th International Conference on Computer Vision Theory and Applications, VISAPP , 2014 . E. Chatzilari, S. Nikolopoulos, Y. Kompatsiaris, and J. Kittler. Active learning in social context for image classification. In 9th International Conference on Computer Vision Theory and Applications, VISAPP, 2014."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022673506211"},{"key":"e_1_3_2_1_8_1","volume-title":"Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm. Applied Statistics, 28(1)","author":"Dawid A. P.","year":"1979","unstructured":"A. P. Dawid and A. M. Skene . Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm. Applied Statistics, 28(1) , 1979 . A. P. Dawid and A. M. Skene. Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm. Applied Statistics, 28(1), 1979."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2187836.2187900"},{"key":"e_1_3_2_1_10_1","volume-title":"IIWeb","author":"Doan A.","year":"2003","unstructured":"A. Doan , Y. Lu , Y. Lee , and J. Han . Object Matching for Information Integration: A Profiler-Based Approach . In IIWeb , 2003 . A. Doan, Y. Lu, Y. Lee, and J. Han. Object Matching for Information Integration: A Profiler-Based Approach. In IIWeb, 2003."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458165"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2396761.2398554"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1066157.1066168"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1656274.1656278"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1009761603038"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-69534-9_41"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837885.1837906"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/2145432.2145597"},{"key":"e_1_3_2_1_19_1","volume-title":"Human Computation","author":"Lease M.","year":"2011","unstructured":"M. Lease . On quality control and machine learning in crowdsourcing . In Human Computation , 2011 . M. Lease. On quality control and machine learning in crowdsourcing. In Human Computation, 2011."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-13094-6_25"},{"key":"e_1_3_2_1_21_1","volume-title":"SWAP","author":"Morris A.","year":"2008","unstructured":"A. Morris , Y. Velegrakis , and P. Bouquet . Entity Identification on the Semantic Web . In SWAP , 2008 . A. Morris, Y. Velegrakis, and P. Bouquet. Entity Identification on the Semantic Web. In SWAP, 2008."},{"key":"e_1_3_2_1_22_1","first-page":"11","author":"Raykar V. C.","year":"2010","unstructured":"V. C. Raykar , S. Yu , L. H. Zhao , G. H. Valadez , C. Florin , L. Bogoni , and L. Moy . Learning from crowds. Journal of Machine Learning Research , 11 , 2010 . V. C. Raykar, S. Yu, L. H. Zhao, G. H. Valadez, C. Florin, L. Bogoni, and L. Moy. Learning from crowds. Journal of Machine Learning Research, 11, 2010.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775087"},{"key":"e_1_3_2_1_24_1","volume-title":"University of Wisconsin","author":"Settles B.","year":"2010","unstructured":"B. Settles . Active learning literature survey . University of Wisconsin , Madison , 2010 . B. Settles. Active learning literature survey. University of Wisconsin, Madison, 2010."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1401890.1401965"},{"key":"e_1_3_2_1_26_1","volume-title":"First AAAI Conference on Human Computation and Crowdsourcing","author":"Sheshadri A.","year":"2013","unstructured":"A. Sheshadri and M. Lease . Square: A benchmark for research on computing crowd consensus . In First AAAI Conference on Human Computation and Crowdsourcing , 2013 . A. Sheshadri and M. Lease. Square: A benchmark for research on computing crowd consensus. In First AAAI Conference on Human Computation and Crowdsourcing, 2013."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/1613715.1613751"},{"key":"e_1_3_2_1_28_1","volume-title":"ICML","author":"Tamuz O.","year":"2011","unstructured":"O. Tamuz , C. Liu , S. Belongie , O. Shamir , A. Kalai , and A. Kalai . Adaptively learning the crowd kernel . In ICML , 2011 . O. Tamuz, C. Liu, S. Belongie, O. Shamir, A. Kalai, and A. Kalai. Adaptively learning the crowd kernel. In ICML, 2011."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995430"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1646396.1646398"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.14778\/2350229.2350263"},{"key":"e_1_3_2_1_32_1","volume-title":"Advances in Neural Information Processing Systems","author":"Welinder P.","year":"2010","unstructured":"P. Welinder , S. Branson , P. Perona , and S. J. Belongie . The multidimensional wisdom of crowds . In Advances in Neural Information Processing Systems , 2010 . P. Welinder, S. Branson, P. Perona, and S. J. Belongie. The multidimensional wisdom of crowds. In Advances in Neural Information Processing Systems, 2010."},{"key":"e_1_3_2_1_33_1","volume-title":"Advances in neural information processing systems","author":"Whitehill J.","year":"2009","unstructured":"J. Whitehill , T.-f. Wu , J. Bergsma , J. R. Movellan , and P. L. Ruvolo . Whose vote should count more: Optimal integration of labels from labelers of unknown expertise . In Advances in neural information processing systems , 2009 . J. Whitehill, T.-f. Wu, J. Bergsma, J. R. Movellan, and P. L. Ruvolo. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In Advances in neural information processing systems, 2009."},{"key":"e_1_3_2_1_34_1","volume-title":"Proceedings of the 28th International Conference on Machine Learning (ICML-11)","author":"Yan Y.","year":"2011","unstructured":"Y. Yan , G. M. Fung , R. Rosales , and J. G. Dy . Active learning from crowds . In Proceedings of the 28th International Conference on Machine Learning (ICML-11) , 2011 . Y. Yan, G. M. Fung, R. Rosales, and J. G. Dy. Active learning from crowds. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), 2011."},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835449.1835526"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/CSE.2009.395"},{"key":"e_1_3_2_1_37_1","volume-title":"Privacy, security, risk and trust (passat)","author":"Zhao L.","year":"2011","unstructured":"L. Zhao , G. Sukthankar , and R. Sukthankar . Incremental relabeling for active learning with noisy crowdsourced annotations . In Privacy, security, risk and trust (passat) , 2011 ieee third international conference on and 2011 ieee third international conference on social computing (socialcom). IEEE , 2011. L. Zhao, G. Sukthankar, and R. Sukthankar. Incremental relabeling for active learning with noisy crowdsourced annotations. In Privacy, security, risk and trust (passat), 2011 ieee third international conference on and 2011 ieee third international conference on social computing (socialcom). IEEE, 2011."}],"event":{"name":"WIMS '14: 4th International Conference on Web Intelligence, Mining and Semantics","sponsor":["Aristotle University of Thessaloniki"],"location":"Thessaloniki Greece","acronym":"WIMS '14"},"container-title":["Proceedings of the 4th International Conference on Web Intelligence, Mining and Semantics (WIMS14)"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2611040.2611047","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2611040.2611047","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T06:55:48Z","timestamp":1750229748000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2611040.2611047"}},"subtitle":["Employing Crowdsourcing for Active Learning"],"short-title":[],"issued":{"date-parts":[[2014,6,2]]},"references-count":37,"alternative-id":["10.1145\/2611040.2611047","10.1145\/2611040"],"URL":"https:\/\/doi.org\/10.1145\/2611040.2611047","relation":{},"subject":[],"published":{"date-parts":[[2014,6,2]]},"assertion":[{"value":"2014-06-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}