{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T01:09:00Z","timestamp":1777338540739,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":28,"publisher":"ACM","license":[{"start":{"date-parts":[[2009,11,2]],"date-time":"2009-11-02T00:00:00Z","timestamp":1257120000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2009,11,2]]},"DOI":"10.1145\/1645953.1646033","type":"proceedings-article","created":{"date-parts":[[2009,11,10]],"date-time":"2009-11-10T18:36:45Z","timestamp":1257878205000},"page":"621-630","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":534,"title":["Expected reciprocal rank for graded relevance"],"prefix":"10.1145","author":[{"given":"Olivier","family":"Chapelle","sequence":"first","affiliation":[{"name":"Yahoo! Labs, Santa Clara, CA, USA"}]},{"given":"Donald","family":"Metlzer","sequence":"additional","affiliation":[{"name":"Yahoo! Labs, Santa Clara, CA, USA"}]},{"given":"Ya","family":"Zhang","sequence":"additional","affiliation":[{"name":"Yahoo! Labs, Sunnyvale, CA, USA"}]},{"given":"Pierre","family":"Grinspan","sequence":"additional","affiliation":[{"name":"Google Inc, San Bruno, CA, USA"}]}],"member":"320","published-online":{"date-parts":[[2009,11,2]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1498759.1498766"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277902"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1321440.1321529"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102363"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277755"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148219"},{"key":"e_1_3_2_1_7_1","volume-title":"Proc. 21st Proc. of Advances in Neural Information Processing Systems","author":"Carterette B.","year":"2007","unstructured":"B. Carterette and R. Jones . Evaluating search engines by modeling the relationship between relevance and clicks . In Proc. 21st Proc. of Advances in Neural Information Processing Systems , 2007 . B. Carterette and R. Jones. Evaluating search engines by modeling the relationship between relevance and clicks. In Proc. 21st Proc. of Advances in Neural Information Processing Systems, 2007."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1526709.1526711"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390334.1390446"},{"key":"e_1_3_2_1_10_1","volume-title":"Report on the Testing and Analysis of an Investigation into the Comparative Efficiency of Indexing Systems","author":"Cleverdon C. W.","year":"1962","unstructured":"C. W. Cleverdon . Report on the Testing and Analysis of an Investigation into the Comparative Efficiency of Indexing Systems . Cranfield College of Aeronautics , 1962 . C. W. Cleverdon. Report on the Testing and Analysis of an Investigation into the Comparative Efficiency of Indexing Systems. Cranfield College of Aeronautics, 1962."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.5090190108"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1341531.1341545"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176350369"},{"key":"e_1_3_2_1_14_1","volume-title":"Overview of the TREC 2002 novelty track. In Proc. 11th Text REtrieval Conference","author":"Harman D.","year":"2002","unstructured":"D. Harman . Overview of the TREC 2002 novelty track. In Proc. 11th Text REtrieval Conference , 2002 . D. Harman. Overview of the TREC 2002 novelty track. In Proc. 11th Text REtrieval Conference, 2002."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2005.01.004"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1416950.1416952"},{"key":"e_1_3_2_1_18_1","volume-title":"Proc. 7th Text REtrieval Conference","author":"Over P.","year":"1998","unstructured":"P. Over . TREC-7 interactive track report . In Proc. 7th Text REtrieval Conference , 1998 . P. Over. TREC-7 interactive track report. In Proc. 7th Text REtrieval Conference, 1998."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458092"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1242572.1242643"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277756"},{"key":"e_1_3_2_1_22_1","volume-title":"Proceedings of the Second Workshop on Evaluating Information Access (EVIA)","author":"Sakai T.","year":"2008","unstructured":"T. Sakai and S. Robertson . Modelling a user population for designing information retrieval metrics . In Proceedings of the Second Workshop on Evaluating Information Access (EVIA) , 2008 . T. Sakai and S. Robertson. Modelling a user population for designing information retrieval metrics. In Proceedings of the Second Workshop on Evaluating Information Access (EVIA), 2008."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.v58:13"},{"issue":"2","key":"e_1_3_2_1_24_1","article-title":"Evaluation of web-based search engines using user-effort measures","volume":"13","author":"Tang M.-C.","year":"2003","unstructured":"M.-C. Tang and Y. Sun . Evaluation of web-based search engines using user-effort measures . Library and Information Science Research Electronic Journal , 13 ( 2 ), 2003 . M.-C. Tang and Y. Sun. Evaluation of web-based search engines using user-effort measures. Library and Information Science Research Electronic Journal, 13(2), 2003.","journal-title":"Library and Information Science Research Electronic Journal"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.5555\/648264.753539"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.6028\/NIST.SP.500-242"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/860435.860440"},{"key":"e_1_3_2_1_28_1","first-page":"7","volume-title":"SIGIR Workshop on Beyond Binary Relevance: Preferences, Diversity, and Set-Level Judgments","author":"Zhou K.","year":"2008","unstructured":"K. Zhou , H. Zha , G. Xue , and Y. Yu . Learning the gain values and discount factors of DCG. In P. N. Bennett, B. Carterette, O. Chapelle, and T. Joachims, editors , SIGIR Workshop on Beyond Binary Relevance: Preferences, Diversity, and Set-Level Judgments , pages 7 -- 14 , 2008 . K. Zhou, H. Zha, G. Xue, and Y. Yu. Learning the gain values and discount factors of DCG. In P. N. Bennett, B. Carterette, O. Chapelle, and T. Joachims, editors, SIGIR Workshop on Beyond Binary Relevance: Preferences, Diversity, and Set-Level Judgments, pages 7--14, 2008."}],"event":{"name":"CIKM '09: Conference on Information and Knowledge Management","location":"Hong Kong China","acronym":"CIKM '09","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 18th ACM conference on Information and knowledge management"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1645953.1646033","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1645953.1646033","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T12:41:10Z","timestamp":1750250470000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1645953.1646033"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2009,11,2]]},"references-count":28,"alternative-id":["10.1145\/1645953.1646033","10.1145\/1645953"],"URL":"https:\/\/doi.org\/10.1145\/1645953.1646033","relation":{},"subject":[],"published":{"date-parts":[[2009,11,2]]},"assertion":[{"value":"2009-11-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}