{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,5]],"date-time":"2025-10-05T19:49:40Z","timestamp":1759693780413,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":24,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,10,19]],"date-time":"2020-10-19T00:00:00Z","timestamp":1603065600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,10,19]]},"DOI":"10.1145\/3340531.3412109","type":"proceedings-article","created":{"date-parts":[[2020,10,19]],"date-time":"2020-10-19T07:01:44Z","timestamp":1603090904000},"page":"2033-2036","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Estimating Topic Difficulty Using Normalized Discounted Cumulated Gain"],"prefix":"10.1145","author":[{"given":"Lukas","family":"Gienapp","sequence":"first","affiliation":[{"name":"Leipzig University, Leipzig, Germany"}]},{"given":"Benno","family":"Stein","sequence":"additional","affiliation":[{"name":"Bauhaus-Universit\u00e4t Weimar, Weimar, Germany"}]},{"given":"Matthias","family":"Hagen","sequence":"additional","affiliation":[{"name":"Martin-Luther-Universit\u00e4t Halle-Wittenberg, Halle (Saale), Germany"}]},{"given":"Martin","family":"Potthast","sequence":"additional","affiliation":[{"name":"Leipzig University, Leipzig, Germany"}]}],"member":"320","published-online":{"date-parts":[[2020,10,19]]},"reference":[{"key":"e_1_3_2_2_1_1","volume-title":"Proc. of ECIR","author":"Aslam J. A.","year":"2007","unstructured":"J. A. Aslam and V. Pavlu . 2007. Query Hardness Estimation Using Jensen-Shannon Divergence Among Multiple Scoring Functions . In Proc. of ECIR 2007 . Springer, 198--209. J. A. Aslam and V. Pavlu. 2007. Query Hardness Estimation Using Jensen-Shannon Divergence Among Multiple Scoring Functions. In Proc. of ECIR 2007. Springer, 198--209."},{"key":"e_1_3_2_2_2_1","volume-title":"Proc. of SIGIR","author":"Carmel D.","year":"2010","unstructured":"D. Carmel and E. Yom-Tov . 2010. Estimating the query difficulty for information retrieval . In Proc. of SIGIR 2010 . ACM, 911. D. Carmel and E. Yom-Tov. 2010. Estimating the query difficulty for information retrieval. In Proc. of SIGIR 2010. ACM, 911."},{"key":"e_1_3_2_2_3_1","volume-title":"Proc. of TREC","author":"Carterette B.","year":"2009","unstructured":"B. Carterette , V. Pavlu , H. Fang , and E. Kanoulas . 2009. Overview of the trec 2009 million query track . In Proc. of TREC 2009 . B. Carterette, V. Pavlu, H. Fang, and E. Kanoulas. 2009. Overview of the trec 2009 million query track. In Proc. of TREC 2009."},{"key":"e_1_3_2_2_4_1","volume-title":"Proc. of TREC","author":"Clarke C. L. A.","year":"2012","unstructured":"C. L. A. Clarke , N. Craswell , and E. M.. Voorhees . 2012 . Overview of the TREC 2012 Web Track . In Proc. of TREC 2012. C. L. A. Clarke, N. Craswell, and E. M.. Voorhees. 2012. Overview of the TREC 2012 Web Track. In Proc. of TREC 2012."},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2970398.2970431"},{"key":"e_1_3_2_2_6_1","volume-title":"Proc. of LREC","author":"Eguchi K.","year":"2002","unstructured":"K. Eguchi , K. Kuriyama , and N. Kando . 2002. Sensitivity of IR systems Evaluation to Topic Difficulty . In Proc. of LREC 2002 . ELRA, 585--589. K. Eguchi, K. Kuriyama, and N. Kando. 2002. Sensitivity of IR systems Evaluation to Topic Difficulty. In Proc. of LREC 2002. ELRA, 585--589."},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3310364"},{"key":"e_1_3_2_2_8_1","volume-title":"Actes de SIGIR","volume":"5","author":"Grivolla J.","year":"2005","unstructured":"J. Grivolla , P. Jourlin , and R. de Mori . 2005 . Automatic classification of queries by expected retrieval performance . Actes de SIGIR , Vol. 5 (2005). J. Grivolla, P. Jourlin, and R. de Mori. 2005. Automatic classification of queries by expected retrieval performance. Actes de SIGIR, Vol. 5 (2005)."},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_2_10_1","volume-title":"Proceedings of CIKM","author":"Kanoulas E.","year":"2009","unstructured":"E. Kanoulas and J. A. Aslam . 2009. Empirical justification of the gain and discount function for nDCG . In Proceedings of CIKM 2009 . ACM, 611--620. E. Kanoulas and J. A. Aslam. 2009. Empirical justification of the gain and discount function for nDCG. In Proceedings of CIKM 2009. ACM, 611--620."},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277824"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-45068-6_1"},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/KSE.2019.8919433"},{"key":"e_1_3_2_2_14_1","volume-title":"Proc. of SPIRE","author":"Iglesias J.","year":"2010","unstructured":"J. P\u00e9 rez- Iglesias and L. Araujo . 2010. Standard Deviation as a Query Hardness Estimator . In Proc. of SPIRE 2010 . Springer, 207--212. J. P\u00e9 rez-Iglesias and L. Araujo. 2010. Standard Deviation as a Query Hardness Estimator. In Proc. of SPIRE 2010. Springer, 207--212."},{"key":"e_1_3_2_2_15_1","volume-title":"Proc. of SIGIR","author":"Raiber F.","year":"2014","unstructured":"F. Raiber and O. Kurland . 2014. Query-performance prediction: setting the expectations straight . In Proc. of SIGIR 2014 . ACM, 13--22. F. Raiber and O. Kurland. 2014. Query-performance prediction: setting the expectations straight. In Proc. of SIGIR 2014. ACM, 13--22."},{"key":"e_1_3_2_2_16_1","volume-title":"Proc. of ECIR","author":"Roitero K.","year":"2017","unstructured":"K. Roitero , E. Maddalena , and S. Mizzaro . 2017. Do Easy Topics Predict Effectiveness Better Than Difficult Topics? . In Proc. of ECIR 2017 . Springer, 605--611. K. Roitero, E. Maddalena, and S. Mizzaro. 2017. Do Easy Topics Predict Effectiveness Better Than Difficult Topics?. In Proc. of ECIR 2017. Springer, 605--611."},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2006.07.020"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(92)90002-H"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2180868.2180873"},{"key":"e_1_3_2_2_20_1","volume-title":"Proc. of INEX","author":"Vercoustre A.-M.","year":"2008","unstructured":"A.-M. Vercoustre , J. Pehcevski , and V. Naumovski . 2008. Topic Difficulty Prediction in Entity Ranking . In Proc. of INEX 2008 . Springer, 280--291. A.-M. Vercoustre, J. Pehcevski, and V. Naumovski. 2008. Topic Difficulty Prediction in Entity Ranking. In Proc. of INEX 2008. Springer, 280--291."},{"key":"e_1_3_2_2_21_1","volume-title":"et almbox","author":"Voorhees E. M.","year":"2005","unstructured":"E. M. Voorhees , D. K. Harman , et almbox . 2005 . TREC : Experiment and evaluation in information retrieval. Vol. 63 . MIT Press , Cambridge. E. M. Voorhees, D. K. Harman, et almbox. 2005. TREC: Experiment and evaluation in information retrieval. Vol. 63. MIT Press, Cambridge."},{"key":"e_1_3_2_2_22_1","volume-title":"Proc. of COLT","author":"Wang Y.","year":"2013","unstructured":"Y. Wang , L. Wang , Y. Li , D. He , and T.-Y. Liu . 2013 . A Theoretical Analysis of NDCG Type Ranking Measures . In Proc. of COLT 2013. JMLR.org, 25--54. Y. Wang, L. Wang, Y. Li, D. He, and T.-Y. Liu. 2013. A Theoretical Analysis of NDCG Type Ranking Measures. In Proc. of COLT 2013. JMLR.org, 25--54."},{"key":"e_1_3_2_2_23_1","volume-title":"Proc. of SIGIR","author":"Yilmaz E.","year":"2008","unstructured":"E. Yilmaz , E. Kanoulas , and J. A. Aslam . 2008. A simple and efficient sampling method for estimating AP and NDCG . In Proc. of SIGIR 2008 . ACM, 603--610. E. Yilmaz, E. Kanoulas, and J. A. Aslam. 2008. A simple and efficient sampling method for estimating AP and NDCG. In Proc. of SIGIR 2008. ACM, 603--610."},{"key":"e_1_3_2_2_24_1","volume-title":"Proc. of SIGIR","author":"Zhu J.","year":"2009","unstructured":"J. Zhu , J. Wang , V. Vinay , and I. J. Cox . 2009. Topic (query) selection for IR evaluation . In Proc. of SIGIR 2009 . ACM, 802--803. J. Zhu, J. Wang, V. Vinay, and I. J. Cox. 2009. Topic (query) selection for IR evaluation. In Proc. of SIGIR 2009. ACM, 802--803."}],"event":{"name":"CIKM '20: The 29th ACM International Conference on Information and Knowledge Management","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGIR ACM Special Interest Group on Information Retrieval"],"location":"Virtual Event Ireland","acronym":"CIKM '20"},"container-title":["Proceedings of the 29th ACM International Conference on Information & Knowledge Management"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3340531.3412109","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3340531.3412109","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:02:30Z","timestamp":1750197750000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3340531.3412109"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,19]]},"references-count":24,"alternative-id":["10.1145\/3340531.3412109","10.1145\/3340531"],"URL":"https:\/\/doi.org\/10.1145\/3340531.3412109","relation":{},"subject":[],"published":{"date-parts":[[2020,10,19]]},"assertion":[{"value":"2020-10-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}