{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:35:57Z","timestamp":1750221357423,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":30,"publisher":"ACM","license":[{"start":{"date-parts":[[2018,2,2]],"date-time":"2018-02-02T00:00:00Z","timestamp":1517529600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Alon grant"},{"name":"ISF grant","award":["1764\/15"],"award-info":[{"award-number":["1764\/15"]}]},{"name":"HUJI Cyber Security Research Center in conjunction with the Israel National Cyber Bureau in the Prime Minister's Office"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2018,2,2]]},"DOI":"10.1145\/3159652.3159670","type":"proceedings-article","created":{"date-parts":[[2018,2,6]],"date-time":"2018-02-06T18:12:23Z","timestamp":1517940743000},"page":"234-242","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Ballpark Crowdsourcing"],"prefix":"10.1145","author":[{"given":"Tom","family":"Hope","sequence":"first","affiliation":[{"name":"Hebrew University of Jerusalem, Jerusalem, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dafna","family":"Shahaf","sequence":"additional","affiliation":[{"name":"Hebrew University of Jerusalem, Jerusalem, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2018,2,2]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Richard Berk Lawrence Sherman Geoffrey Barnes Ellen Kurtz and Lindsay Ahlman . 2009. Forecasting murder within a population of probationers and parolees: a high stakes application of statistical learning. Journal of the Royal Statistical Society: Series A(2009).  Richard Berk Lawrence Sherman Geoffrey Barnes Ellen Kurtz and Lindsay Ahlman . 2009. Forecasting murder within a population of probationers and parolees: a high stakes application of statistical learning. Journal of the Royal Statistical Society: Series A(2009).","DOI":"10.1111\/j.1467-985X.2008.00556.x"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"crossref","unstructured":"M Keith Chen and Jesse M Shapiro . 2007. Do harsher prison conditions reduce recidivism? A discontinuity-based approach. American Law and Economics Review(2007).  M Keith Chen and Jesse M Shapiro . 2007. Do harsher prison conditions reduce recidivism? A discontinuity-based approach. American Law and Economics Review(2007).","DOI":"10.1093\/aler\/ahm006"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2015.03.008"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623703"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/358669.358692"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1017\/S026988890999035X"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824136"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"crossref","unstructured":"Uriel Haran Don A Moore and Carey K Morewedge . 2010. A simple remedy for overprecision in judgment. Judgment and Decision Making(2010).  Uriel Haran Don A Moore and Carey K Morewedge . 2010. A simple remedy for overprecision in judgment. Judgment and Decision Making(2010).","DOI":"10.1037\/e615882011-200"},{"volume-title":"Ballpark Learning: Estimating Labels from Rough Group Comparisons ECML-PKDD.","year":"2016","author":"Hope Tom","key":"e_1_3_2_1_9_1"},{"key":"e_1_3_2_1_10_1","unstructured":"Neil Howe . 2015. Why Millennials Are Texting More And Talking Less http:\/\/www.forbes.com\/sites\/neilhowe\/2015\/07\/15\/why-millennials-are-texting-more-and-talking-less\/. (2015).  Neil Howe . 2015. Why Millennials Are Texting More And Talking Less http:\/\/www.forbes.com\/sites\/neilhowe\/2015\/07\/15\/why-millennials-are-texting-more-and-talking-less\/. (2015)."},{"key":"e_1_3_2_1_11_1","unstructured":"Kaizhu Huang Haiqin Yang Irwin King and Michael R Lyu Learning classifiers from imbalanced data based on biased minimax probability machine Computer Vision and Pattern Recognition 2004.   Kaizhu Huang Haiqin Yang Irwin King and Michael R Lyu Learning classifiers from imbalanced data based on biased minimax probability machine Computer Vision and Pattern Recognition 2004."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"crossref","unstructured":"Peter J Huber . 2011. Robust statistics.  Peter J Huber . 2011. Robust statistics.","DOI":"10.1007\/978-3-642-04898-2_594"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1008636108"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"crossref","unstructured":"Dario Maestripieri . 2014. Night owl women are similar to men in their relationship orientation risk-taking propensities and cortisol levels: Implications for the adaptive significance and evolution of eveningness. Evolutionary Psychology(2014).  Dario Maestripieri . 2014. Night owl women are similar to men in their relationship orientation risk-taking propensities and cortisol levels: Implications for the adaptive significance and evolution of eveningness. Evolutionary Psychology(2014).","DOI":"10.1177\/147470491401200111"},{"volume-title":"Snyder","year":"2010","author":"Cooper Alexia D.","key":"e_1_3_2_1_15_1"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Saemi Park and David V Budescu . 2015. Aggregating multiple probability intervals to improve calibration. Judgment and Decision Making(2015).  Saemi Park and David V Budescu . 2015. Aggregating multiple probability intervals to improve calibration. Judgment and Decision Making(2015).","DOI":"10.1017\/S1930297500003910"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"crossref","unstructured":"Dravzen Prelec H Sebastian Seung and John McCoy . 2017. A solution to the single-question crowd wisdom problem. Nature(2017).  Dravzen Prelec H Sebastian Seung and John McCoy . 2017. A solution to the single-question crowd wisdom problem. Nature(2017).","DOI":"10.1038\/nature21054"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"crossref","unstructured":"Novi Quadrianto Alex J Smola Tiberio S Caetano and Quoc V Le . 2009. Estimating labels from label proportions. JMLR(2009).   Novi Quadrianto Alex J Smola Tiberio S Caetano and Quoc V Le . 2009. Estimating labels from label proportions. JMLR(2009).","DOI":"10.1145\/1390156.1390254"},{"key":"e_1_3_2_1_19_1","unstructured":"Cyrus Rashtchian Peter Young Micah Hodosh and Julia Hockenmaier . 2010. Collecting image annotations using Amazon's Mechanical Turk NAACL HLT.   Cyrus Rashtchian Peter Young Micah Hodosh and Julia Hockenmaier . 2010. Collecting image annotations using Amazon's Mechanical Turk NAACL HLT."},{"volume-title":"SVM Classifier Estimation from Group Probabilities ICML '10","author":"Stefan Rueping","key":"e_1_3_2_1_20_1"},{"volume-title":"Predicting Recidivism In North Carolina","year":"1978","author":"Schmidt Peter","key":"e_1_3_2_1_21_1"},{"volume-title":"Understanding machine learning: From theory to algorithms","author":"Shalev-Shwartz Shai","key":"e_1_3_2_1_22_1","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781107298019"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"crossref","unstructured":"Louis L Thurstone . 1927. A law of comparative judgment. Psychological review(1927).  Louis L Thurstone . 1927. A law of comparative judgment. Psychological review(1927).","DOI":"10.1037\/h0070288"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2764468.2764529"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2566486.2567989"},{"volume-title":"Statlib project repository","year":"2000","author":"P Vlachos","key":"e_1_3_2_1_26_1"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"crossref","unstructured":"Carl Vondrick Deva Ramanan and Donald Patterson . 2010. Efficiently scaling up video annotation with crowdsourced marketplaces European Conference on Computer Vision.   Carl Vondrick Deva Ramanan and Donald Patterson . 2010. Efficiently scaling up video annotation with crowdsourced marketplaces European Conference on Computer Vision.","DOI":"10.1007\/978-3-642-15561-1_44"},{"key":"e_1_3_2_1_28_1","unstructured":"Andrew Whalen and Saiwing Yeung . 2015. Using Ground Truths to Improve Wisdom of the Crowd Estimates. CogSci.  Andrew Whalen and Saiwing Yeung . 2015. Using Ground Truths to Improve Wisdom of the Crowd Estimates. CogSci."},{"volume-title":"ICML'13","year":"2013","author":"Yu Felix","key":"e_1_3_2_1_29_1"},{"key":"e_1_3_2_1_30_1","unstructured":"Liyue Zhao Gita Sukthankar and Rahul Sukthankar . 2011. Robust Active Learning Using Crowdsourced Annotations for Activity Recognition.(2011).  Liyue Zhao Gita Sukthankar and Rahul Sukthankar . 2011. Robust Active Learning Using Crowdsourced Annotations for Activity Recognition.(2011)."}],"event":{"name":"WSDM 2018: The Eleventh ACM International Conference on Web Search and Data Mining","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data","SIGIR ACM Special Interest Group on Information Retrieval"],"location":"Marina Del Rey CA USA","acronym":"WSDM 2018"},"container-title":["Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3159652.3159670","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3159652.3159670","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:26:11Z","timestamp":1750213571000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3159652.3159670"}},"subtitle":["The Wisdom of Rough Group Comparisons"],"short-title":[],"issued":{"date-parts":[[2018,2,2]]},"references-count":30,"alternative-id":["10.1145\/3159652.3159670","10.1145\/3159652"],"URL":"https:\/\/doi.org\/10.1145\/3159652.3159670","relation":{},"subject":[],"published":{"date-parts":[[2018,2,2]]},"assertion":[{"value":"2018-02-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}