{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T20:04:03Z","timestamp":1778875443153,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":31,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,8,20]],"date-time":"2020-08-20T00:00:00Z","timestamp":1597881600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science Foundation","award":["1464800"],"award-info":[{"award-number":["1464800"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,8,23]]},"DOI":"10.1145\/3394486.3403102","type":"proceedings-article","created":{"date-parts":[[2020,8,20]],"date-time":"2020-08-20T23:03:55Z","timestamp":1597964635000},"page":"599-607","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":30,"title":["Towards Fair Truth Discovery from Biased Crowdsourced Answers"],"prefix":"10.1145","author":[{"given":"Yanying","family":"Li","sequence":"first","affiliation":[{"name":"Stevens Institute of Technology, Hoboken, NJ, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haipei","family":"Sun","sequence":"additional","affiliation":[{"name":"Stevens Institute of Technology, Hoboken, NJ, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wendy Hui","family":"Wang","sequence":"additional","affiliation":[{"name":"Stevens Institute of Technology, Hoboken, NJ, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,8,20]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"Propublica. https:\/\/www.propublica.org\/datastore\/ dataset\/compasrecidivism-risk-score-data-and-analysis.  Propublica. https:\/\/www.propublica.org\/datastore\/ dataset\/compasrecidivism-risk-score-data-and-analysis."},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v28i2.19016"},{"key":"e_1_3_2_1_3_1","first-page":"145","volume-title":"ECAI","author":"Brew A.","year":"2010","unstructured":"A. Brew , D. Greene , and P. Cunningham . Using crowdsourcing and active learning to track sentiment in online media . In ECAI , pages 145 -- 150 , 2010 . A. Brew, D. Greene, and P. Cunningham. Using crowdsourcing and active learning to track sentiment in online media. In ECAI, pages 145--150, 2010."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDMW.2009.83"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-010-0190-x"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1089\/big.2016.0047"},{"key":"e_1_3_2_1_7_1","volume-title":"Maximum likelihood estimation of observer error-rates using the em algorithm. Journal of the Royal Statistical Society: Series C (Applied Statistics), 28(1):20--28","author":"Dawid A. P.","year":"1979","unstructured":"A. P. Dawid and A. M. Skene . Maximum likelihood estimation of observer error-rates using the em algorithm. Journal of the Royal Statistical Society: Series C (Applied Statistics), 28(1):20--28 , 1979 . A. P. Dawid and A. M. Skene. Maximum likelihood estimation of observer error-rates using the em algorithm. Journal of the Royal Statistical Society: Series C (Applied Statistics), 28(1):20--28, 1979."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2187836.2187900"},{"key":"e_1_3_2_1_9_1","volume-title":"The accuracy, fairness, and limits of predicting recidivism. Science advances, 4(1):eaao5580","author":"Dressel J.","year":"2018","unstructured":"J. Dressel and H. Farid . The accuracy, fairness, and limits of predicting recidivism. Science advances, 4(1):eaao5580 , 2018 . J. Dressel and H. Farid. The accuracy, fairness, and limits of predicting recidivism. Science advances, 4(1):eaao5580, 2018."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2090236.2090255"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783311"},{"key":"e_1_3_2_1_12_1","first-page":"2415","volume-title":"Advances in Neural Information Processing Systems","author":"Goh G.","year":"2016","unstructured":"G. Goh , A. Cotter , M. Gupta , and M. P. Friedlander . Satisfying real-world goals with dataset constraints . In Advances in Neural Information Processing Systems , pages 2415 -- 2423 , 2016 . G. Goh, A. Cotter, M. Gupta, and M. P. Friedlander. Satisfying real-world goals with dataset constraints. In Advances in Neural Information Processing Systems, pages 2415--2423, 2016."},{"key":"e_1_3_2_1_13_1","volume-title":"Multimodal crowdsourcing for transcribing handwritten documents","author":"Granell E.","year":"2016","unstructured":"E. Granell and C.-D. Martinez-Hinarejos . Multimodal crowdsourcing for transcribing handwritten documents . IEEE\/ACM transactions on audio, speech, and language processing, 25(2):409--419, 2016 . E. Granell and C.-D. Martinez-Hinarejos. Multimodal crowdsourcing for transcribing handwritten documents. IEEE\/ACM transactions on audio, speech, and language processing, 25(2):409--419, 2016."},{"key":"e_1_3_2_1_14_1","volume-title":"Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016","author":"Hardt M.","year":"2016","unstructured":"M. Hardt , E. Price , and N. Srebro . Equality of opportunity in supervised learning . In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016 , December 5 --10 , 2016 , Barcelona, Spain, pages 3315--3323, 2016. M. Hardt, E. Price, and N. Srebro. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5--10, 2016, Barcelona, Spain, pages 3315--3323, 2016."},{"key":"e_1_3_2_1_15_1","first-page":"3315","volume-title":"Advances in neural information processing systems","author":"Hardt M.","year":"2016","unstructured":"M. Hardt , E. Price , N. Srebro , Equality of opportunity in supervised learning . In Advances in neural information processing systems , pages 3315 -- 3323 , 2016 . M. Hardt, E. Price, N. Srebro, et al. Equality of opportunity in supervised learning. In Advances in neural information processing systems, pages 3315--3323, 2016."},{"key":"e_1_3_2_1_16_1","unstructured":"A. Inc. Amazon mechanical turk. https:\/\/www.mturk.com\/mturk.  A. Inc. Amazon mechanical turk. https:\/\/www.mturk.com\/mturk."},{"key":"e_1_3_2_1_17_1","unstructured":"C. Inc. Data for everyone. https:\/\/www.crowdflower.com\/data-for-everyone\/.  C. Inc. Data for everyone. https:\/\/www.crowdflower.com\/data-for-everyone\/."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/2343576.2343643"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/IC4.2009.4909197"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-011-0463-8"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDMW.2011.83"},{"key":"e_1_3_2_1_22_1","first-page":"1953","volume-title":"Advances in neural information processing systems","author":"Karger D. R.","year":"2011","unstructured":"D. R. Karger , S. Oh , and D. Shah . Iterative learning for reliable crowdsourcing systems . In Advances in neural information processing systems , pages 1953 -- 1961 , 2011 . D. R. Karger, S. Oh, and D. Shah. Iterative learning for reliable crowdsourcing systems. In Advances in neural information processing systems, pages 1953--1961, 2011."},{"key":"e_1_3_2_1_23_1","first-page":"1","volume-title":"8th Innovations in Theoretical Computer Science Conference, ITCS","author":"Kleinberg J. M.","year":"2017","unstructured":"J. M. Kleinberg , S. Mullainathan , and M. Raghavan . Inherent trade-offs in the fair determination of risk scores . In 8th Innovations in Theoretical Computer Science Conference, ITCS , pages 43: 1 -- 43 :23, 2017 . J. M. Kleinberg, S. Mullainathan, and M. Raghavan. Inherent trade-offs in the fair determination of risk scores. In 8th Innovations in Theoretical Computer Science Conference, ITCS, pages 43:1--43:23, 2017."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.14778\/2735496.2735505"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897350.2897352"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-39393-3_32"},{"key":"e_1_3_2_1_27_1","volume-title":"Learning from crowds. Journal of Machine Learning Research, 11(Apr):1297--1322","author":"Raykar V. C.","year":"2010","unstructured":"V. C. Raykar , S. Yu , L. H. Zhao , G. H. Valadez , C. Florin , L. Bogoni , and L. Moy . Learning from crowds. Journal of Machine Learning Research, 11(Apr):1297--1322 , 2010 . V. C. Raykar, S. Yu, L. H. Zhao, G. H. Valadez, C. Florin, L. Bogoni, and L. Moy. Learning from crowds. Journal of Machine Learning Research, 11(Apr):1297--1322, 2010."},{"key":"e_1_3_2_1_28_1","first-page":"2424","volume-title":"Advances in neural information processing systems","author":"Welinder P.","year":"2010","unstructured":"P. Welinder , S. Branson , P. Perona , and S. J. Belongie . The multidimensional wisdom of crowds . In Advances in neural information processing systems , pages 2424 -- 2432 , 2010 . P. Welinder, S. Branson, P. Perona, and S. J. Belongie. The multidimensional wisdom of crowds. In Advances in neural information processing systems, pages 2424--2432, 2010."},{"key":"e_1_3_2_1_29_1","first-page":"2035","volume-title":"Advances in neural information processing systems","author":"Whitehill J.","year":"2009","unstructured":"J. Whitehill , T.-f. Wu , J. Bergsma , J. R. Movellan , and P. L. Ruvolo . Whose vote should count more: Optimal integration of labels from labelers of unknown expertise . In Advances in neural information processing systems , pages 2035 -- 2043 , 2009 . J. Whitehill, T.-f. Wu, J. Bergsma, J. R. Movellan, and P. L. Ruvolo. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In Advances in neural information processing systems, pages 2035--2043, 2009."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052660"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.14778\/3055540.3055547"}],"event":{"name":"KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","location":"Virtual Event CA USA","acronym":"KDD '20","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"]},"container-title":["Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3394486.3403102","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3394486.3403102","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3394486.3403102","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:31:34Z","timestamp":1750195894000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3394486.3403102"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,20]]},"references-count":31,"alternative-id":["10.1145\/3394486.3403102","10.1145\/3394486"],"URL":"https:\/\/doi.org\/10.1145\/3394486.3403102","relation":{},"subject":[],"published":{"date-parts":[[2020,8,20]]},"assertion":[{"value":"2020-08-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}