{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:10:07Z","timestamp":1750205407381,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":40,"publisher":"ACM","license":[{"start":{"date-parts":[[2019,5,13]],"date-time":"2019-05-13T00:00:00Z","timestamp":1557705600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2019,5,13]]},"DOI":"10.1145\/3308560.3317082","type":"proceedings-article","created":{"date-parts":[[2019,5,13]],"date-time":"2019-05-13T12:17:59Z","timestamp":1557749879000},"page":"1111-1120","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Learning to Predict Population-Level Label Distributions"],"prefix":"10.1145","author":[{"given":"Tong","family":"Liu","sequence":"first","affiliation":[{"name":"Rochester Institute of Technology"}]},{"given":"Akash","family":"Venkatachalam","sequence":"additional","affiliation":[{"name":"Rochester Institute of Technology"}]},{"given":"Pratik","family":"Sanjay Bongale","sequence":"additional","affiliation":[{"name":"Rochester Institute of Technology"}]},{"given":"Christopher","family":"Homan","sequence":"additional","affiliation":[{"name":"Rochester Institute of Technology"}]}],"member":"320","published-online":{"date-parts":[[2019,5,13]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022873112823"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.15346\/hc.v1i1.3"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-016-9174-x"},{"key":"e_1_3_2_1_4_1","first-page":"993","article-title":"Latent dirichlet allocation","author":"Blei M","year":"2003","unstructured":"David\u00a0 M Blei , Andrew\u00a0 Y Ng , and Michael\u00a0 I Jordan . 2003 . Latent dirichlet allocation . Journal of machine Learning research 3 , Jan (2003), 993 \u2013 1022 . David\u00a0M Blei, Andrew\u00a0Y Ng, and Michael\u00a0I Jordan. 2003. Latent dirichlet allocation. Journal of machine Learning research 3, Jan (2003), 993\u20131022.","journal-title":"Journal of machine Learning research 3"},{"volume-title":"Machine Learning, Meta-Reasoning and Logics","author":"Brazdil Pavel","key":"e_1_3_2_1_5_1","unstructured":"Pavel Brazdil and Peter Clark . 1990. Learning from imperfect data . In Machine Learning, Meta-Reasoning and Logics . Springer , 207\u2013232. Pavel Brazdil and Peter Clark. 1990. Learning from imperfect data. In Machine Learning, Meta-Reasoning and Logics. Springer, 207\u2013232."},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5555\/3013545.3013548"},{"key":"e_1_3_2_1_7_1","volume-title":"Mechanical Turk. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing","volume":"1","author":"Callison-Burch Chris","year":"2009","unstructured":"Chris Callison-Burch . 2009 . Fast, Cheap, and Creative: Evaluating Translation Quality Using Amazon\u2019s Mechanical Turk. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing : Volume 1 - Volume 1(EMNLP \u201909). Association for Computational Linguistics, Stroudsburg, PA, USA, 286\u2013295. http:\/\/dl.acm.org\/citation.cfm?id=1699510.1699548 Chris Callison-Burch. 2009. Fast, Cheap, and Creative: Evaluating Translation Quality Using Amazon\u2019s Mechanical Turk. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1(EMNLP \u201909). Association for Computational Linguistics, Stroudsburg, PA, USA, 286\u2013295. http:\/\/dl.acm.org\/citation.cfm?id=1699510.1699548"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3185515"},{"key":"e_1_3_2_1_9_1","volume-title":"Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied statistics","author":"Dawid Alexander\u00a0Philip","year":"1979","unstructured":"Alexander\u00a0Philip Dawid and Allan\u00a0 M Skene . 1979. Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied statistics ( 1979 ), 20\u201328. Alexander\u00a0Philip Dawid and Allan\u00a0M Skene. 1979. Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied statistics (1979), 20\u201328."},{"key":"e_1_3_2_1_10_1","volume-title":"Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied statistics","author":"Dawid Alexander\u00a0Philip","year":"1979","unstructured":"Alexander\u00a0Philip Dawid and Allan\u00a0 M Skene . 1979. Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied statistics ( 1979 ), 20\u201328. Alexander\u00a0Philip Dawid and Allan\u00a0M Skene. 1979. Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied statistics (1979), 20\u201328."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/1866696.1866705"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2017.2689998"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2016.2545658"},{"key":"e_1_3_2_1_14_1","unstructured":"Xin Geng and Peng Hou. 2015. Pre-release Prediction of Crowd Opinion on Movies by Label Distribution Learning.. In IJCAI. 3511\u20133517.   Xin Geng and Peng Hou. 2015. Pre-release Prediction of Crowd Opinion on Movies by Label Distribution Learning.. In IJCAI. 3511\u20133517."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"crossref","unstructured":"Xin Geng and Miaogen Ling. 2017. Soft Video Parsing by Label Distribution Learning.. In AAAI. 1331\u20131337.   Xin Geng and Miaogen Ling. 2017. Soft Video Parsing by Label Distribution Learning.. In AAAI. 1331\u20131337.","DOI":"10.1609\/aaai.v31i1.10729"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR.2014.764"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(94)00094-8"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-3213"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/IEMBS.2004.1403187"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijforecast.2006.03.001"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"crossref","unstructured":"Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882(2014).  Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882(2014).","DOI":"10.3115\/v1\/D14-1181"},{"key":"e_1_3_2_1_22_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980(2014).","author":"Kingma P","year":"2014","unstructured":"Diederik\u00a0 P Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980(2014). Diederik\u00a0P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980(2014)."},{"key":"e_1_3_2_1_23_1","volume-title":"The workshop on Mining Online Health Reports of the 10th ACM Conference on Web Search and Data Mining","author":"Liu Tong","year":"2017","unstructured":"Tong Liu , Qijin Cheng , Christopher Homan , and Vincent Silenzio . 2017 . Learning from Various Labeling Strategies for Suicide-Related Messages on Social Media: An Experimental Study .. In The workshop on Mining Online Health Reports of the 10th ACM Conference on Web Search and Data Mining . Cambridge, UK. https:\/\/arxiv.org\/pdf\/1701.08796.pdf Tong Liu, Qijin Cheng, Christopher Homan, and Vincent Silenzio. 2017. Learning from Various Labeling Strategies for Suicide-Related Messages on Social Media: An Experimental Study.. In The workshop on Mining Online Health Reports of the 10th ACM Conference on Web Search and Data Mining. Cambridge, UK. https:\/\/arxiv.org\/pdf\/1701.08796.pdf"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1099"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btl346"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.5555\/972470.972475"},{"key":"e_1_3_2_1_27_1","volume-title":"Medicine","author":"Engineering","year":"2017","unstructured":"Engineering , National Academies\u00a0of Sciences , Medicine , 2017 . Federal Statistics, Multiple Data Sources, and Privacy Protection: Next Steps. National Academies Press . Engineering, National Academies\u00a0of Sciences, Medicine, 2017. Federal Statistics, Multiple Data Sources, and Privacy Protection: Next Steps. National Academies Press."},{"key":"e_1_3_2_1_28_1","unstructured":"Jordan Pearson. 2016 (accessed November 11 2018). Why An AI-Judged Beauty Contest Picked Nearly All White Winners. https:\/\/motherboard.vice.com\/en_us\/article\/78k7de\/why-an-ai-judged-beauty-contest-picked-nearly-all-white-winners  Jordan Pearson. 2016 (accessed November 11 2018). Why An AI-Judged Beauty Contest Picked Nearly All White Winners. https:\/\/motherboard.vice.com\/en_us\/article\/78k7de\/why-an-ai-judged-beauty-contest-picked-nearly-all-white-winners"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"crossref","unstructured":"Jeffrey Pennington Richard Socher and Christopher\u00a0D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Empirical Methods in Natural Language Processing (EMNLP). 1532\u20131543. http:\/\/www.aclweb.org\/anthology\/D14-1162  Jeffrey Pennington Richard Socher and Christopher\u00a0D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Empirical Methods in Natural Language Processing (EMNLP). 1532\u20131543. http:\/\/www.aclweb.org\/anthology\/D14-1162","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10651-007-0043-y"},{"key":"e_1_3_2_1_31_1","first-page":"2648","article-title":"Sense beauty by label distribution learning","author":"Ren Yi","year":"2017","unstructured":"Yi Ren and Xin Geng . 2017 . Sense beauty by label distribution learning . In Proc, IJCAI. 2648 \u2013 2654 . Yi Ren and Xin Geng. 2017. Sense beauty by label distribution learning. In Proc, IJCAI. 2648\u20132654.","journal-title":"Proc, IJCAI."},{"key":"e_1_3_2_1_32_1","volume-title":"Proceedings of the 1st Workshop on Human-Centered Machine Learning at SIGCHI.","author":"Schaekermann Mike","year":"2016","unstructured":"Mike Schaekermann , Edith Law , Alex\u00a0 C Williams , and William Callaghan . 2016 . Resolvable vs. irresolvable ambiguity: A new hybrid framework for dealing with uncertain ground truth . In Proceedings of the 1st Workshop on Human-Centered Machine Learning at SIGCHI. Mike Schaekermann, Edith Law, Alex\u00a0C Williams, and William Callaghan. 2016. Resolvable vs. irresolvable ambiguity: A new hybrid framework for dealing with uncertain ground truth. In Proceedings of the 1st Workshop on Human-Centered Machine Learning at SIGCHI."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/0167-8655(96)00105-5"},{"key":"e_1_3_2_1_34_1","volume-title":"Inferring ground truth from subjective labelling of venus images. Advances in neural information processing systems 7","author":"Smyth Padhraic","year":"1995","unstructured":"Padhraic Smyth , Usama Fayyad , Michael Burl , Pietro Perona , and Pierre Baldi . 1995. Inferring ground truth from subjective labelling of venus images. Advances in neural information processing systems 7 ( 1995 ), 1085\u20131092. Padhraic Smyth, Usama Fayyad, Michael Burl, Pietro Perona, and Pierre Baldi. 1995. Inferring ground truth from subjective labelling of venus images. Advances in neural information processing systems 7 (1995), 1085\u20131092."},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.5555\/1613715.1613751"},{"key":"e_1_3_2_1_36_1","volume-title":"Microsoft \u2019deeply sorry","author":"Staff","year":"2016","unstructured":"Staff and agencies. 2016 (accessed November 11, 2018). Microsoft \u2019deeply sorry \u2019 for racist and sexist tweets by AI chatbot. https:\/\/www.theguardian.com\/technology\/ 2016 \/mar\/26\/microsoft-deeply-sorry-for-offensive-tweets-by-ai-chatbot Staff and agencies. 2016 (accessed November 11, 2018). Microsoft \u2019deeply sorry\u2019 for racist and sexist tweets by AI chatbot. https:\/\/www.theguardian.com\/technology\/2016\/mar\/26\/microsoft-deeply-sorry-for-offensive-tweets-by-ai-chatbot"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1080\/13658810500286976"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2015.2504974"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2013.39"},{"key":"e_1_3_2_1_40_1","volume-title":"Class noise vs. attribute noise: A quantitative study. Artificial intelligence review 22, 3","author":"Zhu Xingquan","year":"2004","unstructured":"Xingquan Zhu and Xindong Wu. 2004. Class noise vs. attribute noise: A quantitative study. Artificial intelligence review 22, 3 ( 2004 ), 177\u2013210. Xingquan Zhu and Xindong Wu. 2004. Class noise vs. attribute noise: A quantitative study. Artificial intelligence review 22, 3 (2004), 177\u2013210."}],"event":{"name":"WWW '19: The Web Conference","sponsor":["IW3C2 International World Wide Web Conference Committee"],"location":"San Francisco USA","acronym":"WWW '19"},"container-title":["Companion Proceedings of The 2019 World Wide Web Conference"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3308560.3317082","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3308560.3317082","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:53:35Z","timestamp":1750204415000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3308560.3317082"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,13]]},"references-count":40,"alternative-id":["10.1145\/3308560.3317082","10.1145\/3308560"],"URL":"https:\/\/doi.org\/10.1145\/3308560.3317082","relation":{},"subject":[],"published":{"date-parts":[[2019,5,13]]},"assertion":[{"value":"2019-05-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}