{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:14:11Z","timestamp":1750306451605,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":31,"publisher":"ACM","license":[{"start":{"date-parts":[[2015,10,17]],"date-time":"2015-10-17T00:00:00Z","timestamp":1445040000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1421399"],"award-info":[{"award-number":["1421399"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2015,10,17]]},"DOI":"10.1145\/2806416.2806492","type":"proceedings-article","created":{"date-parts":[[2016,12,1]],"date-time":"2016-12-01T19:23:23Z","timestamp":1480620203000},"page":"1391-1400","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Aggregation of Crowdsourced Ordinal Assessments and Integration with Learning to Rank"],"prefix":"10.1145","author":[{"given":"Pavel","family":"Metrikov","sequence":"first","affiliation":[{"name":"Northeastern University, Boston, MA, USA"}]},{"given":"Virgil","family":"Pavlu","sequence":"additional","affiliation":[{"name":"Northeastern University, Boston, MA, USA"}]},{"given":"Javed A.","family":"Aslam","sequence":"additional","affiliation":[{"name":"Northeastern University, Boston, MA, USA"}]}],"member":"320","published-online":{"date-parts":[[2015,10,17]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02293814"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.5555\/1162264"},{"key":"e_1_3_2_1_3_1","volume-title":"From ranknet to lambdarank to lambdamart: An overview","author":"Burges C.J.C.","year":"2010","unstructured":"C.J.C. Burges . From ranknet to lambdarank to lambdamart: An overview , 2010 . C.J.C. Burges. From ranknet to lambdarank to lambdamart: An overview, 2010."},{"key":"e_1_3_2_1_4_1","volume-title":"Overview of the TREC 2010 Relevance Feedback Track (Notebook). In TREC","author":"Buckley M. Lease C.","year":"2010","unstructured":"M. Lease C. Buckley and M. D. Smucker . Overview of the TREC 2010 Relevance Feedback Track (Notebook). In TREC , 2010 . M. Lease C. Buckley and M. D. Smucker. Overview of the TREC 2010 Relevance Feedback Track (Notebook). In TREC, 2010."},{"key":"e_1_3_2_1_5_1","first-page":"1","article-title":"Yahoo! learning to rank challenge overview","volume":"14","author":"Chapelle O.","year":"2011","unstructured":"O. Chapelle and Y. Chang . Yahoo! learning to rank challenge overview . JMLR , 14 : 1 -- 24 , 2011 . O. Chapelle and Y. Chang. Yahoo! learning to rank challenge overview. JMLR, 14:1--24, 2011.","journal-title":"JMLR"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2433396.2433420"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.2307\/2346806"},{"key":"e_1_3_2_1_8_1","volume-title":"Annals of Statistics","author":"Friedman J. H.","year":"2000","unstructured":"J. H. Friedman . Greedy function approximation: A gradient boosting machine . Annals of Statistics , 2000 . J. H. Friedman. Greedy function approximation: A gradient boosting machine. Annals of Statistics, 2000."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2009932"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-28997-2_16"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1996.10476662"},{"key":"e_1_3_2_1_12_1","first-page":"225","volume-title":"ICML","author":"Wang Chao","year":"2012","unstructured":"Chao L. and Y.-M. Wang . Truelabel confusions : A spectrum of probabilistic models in analyzing multiple ratings . In ICML , pages 225 -- 232 , 2012 . Chao L. and Y.-M. Wang. Truelabel confusions: A spectrum of probabilistic models in analyzing multiple ratings. In ICML, pages 225--232, 2012."},{"key":"e_1_3_2_1_13_1","volume-title":"Inferring ground truth from multi-annotator ordinal data: A probabilistic approach. arXiv:1305.0015","author":"Lakshminarayanan B.","year":"2013","unstructured":"B. Lakshminarayanan and Y. W. Teh . Inferring ground truth from multi-annotator ordinal data: A probabilistic approach. arXiv:1305.0015 , 2013 . B. Lakshminarayanan and Y. W. Teh. Inferring ground truth from multi-annotator ordinal data: A probabilistic approach. arXiv:1305.0015, 2013."},{"key":"e_1_3_2_1_14_1","first-page":"692","volume-title":"NIPS","author":"Liu Q.","year":"2012","unstructured":"Q. Liu , J. Peng , and A. T Ihler . Variational inference for crowdsourcing . In NIPS , pages 692 -- 700 . 2012 . Q. Liu, J. Peng, and A. T Ihler. Variational inference for crowdsourcing. In NIPS, pages 692--700. 2012."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02296272"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2499178.2499198"},{"key":"e_1_3_2_1_17_1","unstructured":"Paul Mineiro. Ordered values and mechanical turk. http:\/\/www.machinedlearnings.com 2011.  Paul Mineiro. Ordered values and mechanical turk. http:\/\/www.machinedlearnings.com 2011."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2684822.2685308"},{"key":"e_1_3_2_1_19_1","volume-title":"Numerical Recipes","author":"Press W. H.","year":"2007","unstructured":"W. H. Press , S. A. Teukolsky , W. T. Vetterling , and B. P. Flannery . Numerical Recipes 3 rd Edition: The Art of Scientific Computing . 3 edition, 2007 . W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. Numerical Recipes 3rd Edition: The Art of Scientific Computing. 3 edition, 2007.","edition":"3"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.5555\/1756006.1859894"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11222-009-9125-z"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1401890.1401965"},{"key":"e_1_3_2_1_23_1","volume-title":"Overview of the TREC 2013 Crowdsourcing Track. In TREC","author":"Smucker M.","year":"2014","unstructured":"M. Smucker , G. Kazai , and M. Lease . Overview of the TREC 2013 Crowdsourcing Track. In TREC , 2014 . M. Smucker, G. Kazai, and M. Lease. Overview of the TREC 2013 Crowdsourcing Track. In TREC, 2014."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1321440.1321528"},{"key":"e_1_3_2_1_25_1","volume-title":"Biometrics","author":"Uebersax J. S.","year":"1993","unstructured":"J. S. Uebersax and W. M. Grove . A latent trait finite mixture model for the analysis of rating agreement . In Biometrics , December 1993 . J. S. Uebersax and W. M. Grove. A latent trait finite mixture model for the analysis of rating agreement. In Biometrics, December 1993."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2566486.2567989"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/2627435.2638572"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441846"},{"key":"e_1_3_2_1_29_1","first-page":"2035","volume-title":"NIPS","author":"Whitehill J.","year":"2009","unstructured":"J. Whitehill , P. Ruvolo , T. Wu , J. Bergsma , and J. R. Movellan . Whose vote should count more: Optimal integration of labels from labelers of unknown expertise . In NIPS , pages 2035 -- 2043 , 2009 . J. Whitehill, P. Ruvolo, T. Wu, J. Bergsma, and J. R. Movellan. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In NIPS, pages 2035--2043, 2009."},{"key":"e_1_3_2_1_30_1","volume-title":"ICML","author":"Zhou D.","year":"2014","unstructured":"D. Zhou , Q. Liu , J. C. Platt , and C. Meek . Aggregating ordinal labels from crowds by minimax conditional entropy . In ICML , 2014 . D. Zhou, Q. Liu, J. C. Platt, and C. Meek. Aggregating ordinal labels from crowds by minimax conditional entropy. In ICML, 2014."},{"key":"e_1_3_2_1_31_1","volume-title":"NIPS","author":"Zhou D.","year":"2012","unstructured":"D. Zhou , J. Platt , S. Basu , and Y. Mao . Learning from the wisdom of crowds by minimax entropy . In NIPS , 2012 . D. Zhou, J. Platt, S. Basu, and Y. Mao. Learning from the wisdom of crowds by minimax entropy. In NIPS, 2012."}],"event":{"name":"CIKM'15: 24th ACM International Conference on Information and Knowledge Management","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGIR ACM Special Interest Group on Information Retrieval"],"location":"Melbourne Australia","acronym":"CIKM'15"},"container-title":["Proceedings of the 24th ACM International on Conference on Information and Knowledge Management"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2806416.2806492","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2806416.2806492","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T05:43:11Z","timestamp":1750225391000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2806416.2806492"}},"subtitle":["A Latent Trait Model"],"short-title":[],"issued":{"date-parts":[[2015,10,17]]},"references-count":31,"alternative-id":["10.1145\/2806416.2806492","10.1145\/2806416"],"URL":"https:\/\/doi.org\/10.1145\/2806416.2806492","relation":{},"subject":[],"published":{"date-parts":[[2015,10,17]]},"assertion":[{"value":"2015-10-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}