{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T00:50:57Z","timestamp":1774399857519,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":32,"publisher":"ACM","license":[{"start":{"date-parts":[[2008,7,20]],"date-time":"2008-07-20T00:00:00Z","timestamp":1216512000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2008,7,20]]},"DOI":"10.1145\/1390334.1390446","type":"proceedings-article","created":{"date-parts":[[2008,7,22]],"date-time":"2008-07-22T13:46:39Z","timestamp":1216734399000},"page":"659-666","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":631,"title":["Novelty and diversity in information retrieval evaluation"],"prefix":"10.1145","author":[{"given":"Charles L.A.","family":"Clarke","sequence":"first","affiliation":[{"name":"University of Waterloo, Waterlo, ON, Canada"}]},{"given":"Maheedhar","family":"Kolla","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, ON, Canada"}]},{"given":"Gordon V.","family":"Cormack","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, ON, Canada"}]},{"given":"Olga","family":"Vechtomova","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, ON, Canada"}]},{"given":"Azin","family":"Ashkan","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, ON, Canada"}]},{"given":"Stefan","family":"B\u00fcttcher","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, ON, Canada"}]},{"given":"Ian","family":"MacKinnon","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, ON, Canada"}]}],"member":"320","published-online":{"date-parts":[[2008,7,20]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148177"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148175"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277902"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-24752-4_10"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1099554.1099733"},{"issue":"3","key":"e_1_3_2_1_6_1","first-page":"105","article-title":"A two stage view of relevance and the retrieval process. Information Processing &","volume":"18","author":"Boyce B.","year":"1982","unstructured":"B. Boyce . Beyond topicality : A two stage view of relevance and the retrieval process. Information Processing & Management , 18 ( 3 ): 105 -- 109 , 1982 . B. Boyce. Beyond topicality: A two stage view of relevance and the retrieval process. Information Processing & Management, 18(3):105--109, 1982.","journal-title":"Management"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/792550.792552"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1008992.1009000"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102363"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277755"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.291025"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/366836.366860"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148245"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/564376.564429"},{"key":"e_1_3_2_1_15_1","volume-title":"Overview of the TREC 2006 question answering track. In 15th Text REtrieval Conference","author":"Dang H. T.","year":"2006","unstructured":"H. T. Dang , J. Lin , and D. Kelly . Overview of the TREC 2006 question answering track. In 15th Text REtrieval Conference , Gaithersburg, Maryland , 2006 . H. T. Dang, J. Lin, and D. Kelly. Overview of the TREC 2006 question answering track. In 15th Text REtrieval Conference, Gaithersburg, Maryland, 2006."},{"key":"e_1_3_2_1_16_1","volume-title":"16th International World Wide Web Conference","author":"Dupret G.","year":"2007","unstructured":"G. Dupret , V. Murdock , and B. Piwowarski . Web search engine evaluation using clickthrough data and a user model . In 16th International World Wide Web Conference , 2007 . G. Dupret, V. Murdock, and B. Piwowarski. Web search engine evaluation using clickthrough data and a user model. In 16th International World Wide Web Conference, 2007."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/0020-0271(64)90006-3"},{"key":"e_1_3_2_1_18_1","first-page":"21","volume-title":"The TREC test collections","author":"Harman D. K.","year":"2005","unstructured":"D. K. Harman . The TREC test collections . In Ellen M. Voorhees and Donna K. Harman, editors, TREC : Experiment and Evaluation in Information Retrieval, chapter 2, pages 21 -- 52 . The MIT Press , 2005 . D. K. Harman. The TREC test collections. In Ellen M. Voorhees and Donna K. Harman, editors, TREC: Experiment and Evaluation in Information Retrieval, chapter 2, pages 21--52. The MIT Press, 2005."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2005.11.003"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076063"},{"key":"e_1_3_2_1_22_1","first-page":"1","volume-title":"Language Modeling for Information Retrieval","author":"Lafferty J.","year":"2003","unstructured":"J. Lafferty and C. Zhai . Probabilistic relevance models based on document and query generation . In W. B. Croft and J. Lafferty, editors, Language Modeling for Information Retrieval , chapter 1, pages 1 -- 10 . Kluwer Academic Publishers , 2003 . J. Lafferty and C. Zhai. Probabilistic relevance models based on document and query generation. In W. B. Croft and J. Lafferty, editors, Language Modeling for Information Retrieval, chapter 1, pages 1--10. Kluwer Academic Publishers, 2003."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.v57:7"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1108\/eb026647"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1328964.1328965"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(00)00015-7"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.291017"},{"key":"e_1_3_2_1_28_1","volume-title":"Overview of the TREC 2005 question answering track. In 14th Text REtrieval Conference","author":"Voorhees E. M.","year":"2005","unstructured":"E. M. Voorhees and H. T. Dang . Overview of the TREC 2005 question answering track. In 14th Text REtrieval Conference , Gaithersburg, Maryland , 2005 . E. M. Voorhees and H. T. Dang. Overview of the TREC 2005 question answering track. In 14th Text REtrieval Conference, Gaithersburg, Maryland, 2005."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.v59:2"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277790"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/860435.860440"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2004.11.003"}],"event":{"name":"SIGIR '08: The 31st Annual International ACM SIGIR Conference","location":"Singapore Singapore","acronym":"SIGIR '08","sponsor":["ACM Association for Computing Machinery","SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1390334.1390446","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1390334.1390446","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T14:47:20Z","timestamp":1750258040000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1390334.1390446"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,7,20]]},"references-count":32,"alternative-id":["10.1145\/1390334.1390446","10.1145\/1390334"],"URL":"https:\/\/doi.org\/10.1145\/1390334.1390446","relation":{},"subject":[],"published":{"date-parts":[[2008,7,20]]},"assertion":[{"value":"2008-07-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}