{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,5]],"date-time":"2025-10-05T19:58:53Z","timestamp":1759694333904,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":17,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,10,19]],"date-time":"2020-10-19T00:00:00Z","timestamp":1603065600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,10,19]]},"DOI":"10.1145\/3340531.3412123","type":"proceedings-article","created":{"date-parts":[[2020,10,19]],"date-time":"2020-10-19T07:01:44Z","timestamp":1603090904000},"page":"2037-2040","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["The Impact of Negative Relevance Judgments on NDCG"],"prefix":"10.1145","author":[{"given":"Lukas","family":"Gienapp","sequence":"first","affiliation":[{"name":"Leipzig University, Leipzig, Germany"}]},{"given":"Maik","family":"Fr\u00f6be","sequence":"additional","affiliation":[{"name":"Martin-Luther-Universit\u00e4t Halle-Wittenberg, Halle (Saale), Germany"}]},{"given":"Matthias","family":"Hagen","sequence":"additional","affiliation":[{"name":"Martin-Luther-Universit\u00e4t Halle-Wittenberg, Halle (Saale), Germany"}]},{"given":"Martin","family":"Potthast","sequence":"additional","affiliation":[{"name":"Leipzig University, Leipzig, Germany"}]}],"member":"320","published-online":{"date-parts":[[2020,10,19]]},"reference":[{"key":"e_1_3_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277805"},{"key":"e_1_3_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130348.3130373"},{"volume-title":"Proc. of TREC.","author":"Clarke C.L. A.","key":"e_1_3_2_2_3_1","unstructured":"C.L. A. Clarke , N. Craswell , I. Soboroff , and G. V. Cormack . 2010. Overview of the TREC 2010 Web Track . In Proc. of TREC. C.L. A. Clarke, N. Craswell, I. Soboroff, and G. V. Cormack. 2010. Overview of the TREC 2010 Web Track. In Proc. of TREC."},{"volume-title":"Proc. of TREC.","author":"Clarke C. L. A.","key":"e_1_3_2_2_4_1","unstructured":"C. L. A. Clarke , N. Craswell , I. Soboroff , and E. M. Voorhees . 2011. Overview of the TREC 2011 Web Track . In Proc. of TREC. C. L. A. Clarke, N. Craswell, I. Soboroff, and E. M. Voorhees. 2011. Overview of the TREC 2011 Web Track. In Proc. of TREC."},{"volume-title":"Proc. of TREC.","author":"Clarke C. L. A.","key":"e_1_3_2_2_5_1","unstructured":"C. L. A. Clarke , N. Craswell , and E. M. Voorhees . 2012. Overview of the TREC 2012 Web Track . In Proc. of TREC. C. L. A. Clarke, N. Craswell, and E. M. Voorhees. 2012. Overview of the TREC 2012 Web Track. In Proc. of TREC."},{"volume-title":"Proc. of TREC.","author":"Collins-Thompson K.","key":"e_1_3_2_2_6_1","unstructured":"K. Collins-Thompson , P. Bennett , F. Diaz , C. L. A. Clarke , and E. M. Voorhees . 2013. Overview of the TREC 2013 Web Track . In Proc. of TREC. K. Collins-Thompson, P. Bennett, F. Diaz, C. L. A. Clarke, and E. M. Voorhees. 2013. Overview of the TREC 2013 Web Track. In Proc. of TREC."},{"volume-title":"Proc. of TREC.","author":"Collins-Thompson K.","key":"e_1_3_2_2_7_1","unstructured":"K. Collins-Thompson , C. Macdonald , P. Bennett , F. Diaz , and E. M. Voorhees . 2014. TREC 2014 Web Track Overview . In Proc. of TREC. K. Collins-Thompson, C. Macdonald, P. Bennett, F. Diaz, and E. M. Voorhees. 2014. TREC 2014 Web Track Overview. In Proc. of TREC."},{"key":"e_1_3_2_2_8_1","volume-title":"The Lemur Project-Wiki-RankLib. Lemur Project","author":"Dang V.","year":"2013","unstructured":"V. Dang . 2013. The Lemur Project-Wiki-RankLib. Lemur Project ( 2013 ). Available: https:\/\/sourceforge.net\/p\/lemur\/wiki\/RankLib. V. Dang. 2013. The Lemur Project-Wiki-RankLib. Lemur Project (2013). Available: https:\/\/sourceforge.net\/p\/lemur\/wiki\/RankLib."},{"volume-title":"Proc. of CIKM. ACM.","author":"Gienapp L.","key":"e_1_3_2_2_9_1","unstructured":"L. Gienapp and B. Stein and M. Hagen and M. Potthast . 2020. Estimating Topic Difficulty Using Normalized Discounted Cumulated Gain . In Proc. of CIKM. ACM. L. Gienapp and B. Stein and M. Hagen and M. Potthast. 2020. Estimating Topic Difficulty Using Normalized Discounted Cumulated Gain. In Proc. of CIKM. ACM."},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"volume-title":"Proc. of CIKM. ACM, 611--620","author":"Kanoulas E.","key":"e_1_3_2_2_11_1","unstructured":"E. Kanoulas and J. A. Aslam . 2009. Empirical Justification of the Gain and Discount Function for nDCG . In Proc. of CIKM. ACM, 611--620 . E. Kanoulas and J. A. Aslam. 2009. Empirical Justification of the Gain and Discount Function for nDCG. In Proc. of CIKM. ACM, 611--620."},{"volume-title":"Information Retrieval Technology","author":"Moffat A.","key":"e_1_3_2_2_12_1","unstructured":"A. Moffat . 2013. Seven Numeric Properties of Effectiveness Metrics . In Information Retrieval Technology , Springer , Berlin, Heidelberg , 1--12. A. Moffat. 2013. Seven Numeric Properties of Effectiveness Metrics. In Information Retrieval Technology, Springer, Berlin, Heidelberg, 1--12."},{"volume-title":"TrecTools: An Open-Source Python Library for Information Retrieval Practitioners Involved in TREC-like Campaigns In Proc. of SIGIR. ACM.","author":"Palotti J.","key":"e_1_3_2_2_13_1","unstructured":"J. Palotti , H. Scells , and G. Zuccon . 2019 . TrecTools: An Open-Source Python Library for Information Retrieval Practitioners Involved in TREC-like Campaigns In Proc. of SIGIR. ACM. J. Palotti, H. Scells, and G. Zuccon. 2019. TrecTools: An Open-Source Python Library for Information Retrieval Practitioners Involved in TREC-like Campaigns In Proc. of SIGIR. ACM."},{"key":"e_1_3_2_2_14_1","volume-title":"abs\/1306.2597","author":"Qin T.","year":"2013","unstructured":"T. Qin , T. Liu . 2013. Introducing LETOR 4.0 Datasets . Co RR , Vol . abs\/1306.2597 ( 2013 ). T. Qin, T. Liu. 2013. Introducing LETOR 4.0 Datasets. CoRR, Vol. abs\/1306.2597 (2013)."},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148261"},{"key":"e_1_3_2_2_16_1","volume-title":"et almbox","author":"Voorhees E. M","year":"2005","unstructured":"E. M Voorhees , D. K. Harman , et almbox . 2005 . TREC : Experiment and Evaluation in Information Retrieval. Vol. 63 . MIT press Cambridge . E. M Voorhees, D. K. Harman, et almbox. 2005. TREC: Experiment and Evaluation in Information Retrieval. Vol. 63. MIT press Cambridge."},{"key":"e_1_3_2_2_17_1","volume-title":"Proc. of COLT. 25--54","author":"Wang Y.","year":"2013","unstructured":"Y. Wang , L. Wang , Y. Li , D. He , and T.-Y. Liu . 2013 . A Theoretical Analysis of NDCG Type Ranking Measures . In Proc. of COLT. 25--54 . Y. Wang, L. Wang, Y. Li, D. He, and T.-Y. Liu. 2013. A Theoretical Analysis of NDCG Type Ranking Measures. In Proc. of COLT. 25--54."}],"event":{"name":"CIKM '20: The 29th ACM International Conference on Information and Knowledge Management","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGIR ACM Special Interest Group on Information Retrieval"],"location":"Virtual Event Ireland","acronym":"CIKM '20"},"container-title":["Proceedings of the 29th ACM International Conference on Information & Knowledge Management"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3340531.3412123","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3340531.3412123","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:02:30Z","timestamp":1750197750000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3340531.3412123"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,19]]},"references-count":17,"alternative-id":["10.1145\/3340531.3412123","10.1145\/3340531"],"URL":"https:\/\/doi.org\/10.1145\/3340531.3412123","relation":{},"subject":[],"published":{"date-parts":[[2020,10,19]]},"assertion":[{"value":"2020-10-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}