{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T11:48:55Z","timestamp":1790768935902,"version":"4.1.0"},"reference-count":16,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2015,6,23]],"date-time":"2015-06-23T00:00:00Z","timestamp":1435017600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGIR Forum"],"published-print":{"date-parts":[[2015,6,23]]},"abstract":"<jats:p>We present a non-traditional retrieval problem we call subtopic retrieval. The subtopic retrieval problem is concerned with finding documents that cover many different subtopics of a query topic. In such a problem, the utility of a document in a ranking is dependent on other documents in the ranking, violating the assumption of independent relevance which is assumed in most traditional retrieval methods. Subtopic retrieval poses challenges for evaluating performance, as well as for developing effective algorithms. We propose a framework for evaluating subtopic retrieval which generalizes the traditional precision and recall metrics by accounting for intrinsic topic difficulty as well as redundancy in documents. We propose and systematically evaluate several methods for performing subtopic retrieval using statistical language models and a maximal marginal relevance (MMR) ranking strategy. A mixture model combined with query likelihood relevance ranking is shown to modestly outperform a baseline relevance ranking on a data set used in the TREC interactive track.<\/jats:p>","DOI":"10.1145\/2795403.2795405","type":"journal-article","created":{"date-parts":[[2015,6,25]],"date-time":"2015-06-25T14:36:19Z","timestamp":1435242979000},"page":"2-9","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":39,"title":["Beyond Independent Relevance"],"prefix":"10.1145","volume":"49","author":[{"given":"ChengXiang","family":"Zhai","sequence":"first","affiliation":[{"name":"University of Illinois at Urbana-Champaign"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"William W.","family":"Cohen","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John","family":"Lafferty","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2015,6,23]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/383952.383954"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.291025"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/285055.285059"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of TREC 2002","author":"Harman D.","year":"2002","unstructured":"D. Harman . Overview of the trec 2002 novelty track . In Proceedings of TREC 2002 , 2002 . D. Harman. Overview of the trec 2002 novelty track. In Proceedings of TREC 2002, 2002."},{"key":"e_1_2_1_5_1","first-page":"57","volume-title":"The Seventh Text REtrieval Conference (TREC-8)","author":"Hersh W.","year":"2000","unstructured":"W. Hersh and P. Over . Trec-8 interactive track report. In E. Voorhees and D. Harman, editors , The Seventh Text REtrieval Conference (TREC-8) , pages 57 -- 64 , 2000 . NIST Special Publication 500-246. W. Hersh and P. Over. Trec-8 interactive track report. In E. Voorhees and D. Harman, editors, The Seventh Text REtrieval Conference (TREC-8), pages 57--64, 2000. NIST Special Publication 500-246."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/345508.345545"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/383952.383970"},{"key":"e_1_2_1_8_1","first-page":"103","volume-title":"Proceedings of the 2001 Text REtrieval Conference","author":"Ogilvie P.","year":"2002","unstructured":"P. Ogilvie and J. Callan . Experiments using the lemur toolkit . In Proceedings of the 2001 Text REtrieval Conference , pages 103 -- 108 , 2002 . P. Ogilvie and J. Callan. Experiments using the lemur toolkit. In Proceedings of the 2001 Text REtrieval Conference, pages 103--108, 2002."},{"key":"e_1_2_1_9_1","first-page":"73","volume-title":"The Sixth Text REtrieval Conference (TREC-6)","author":"Over P.","year":"1998","unstructured":"P. Over . Trec-6 interactive track report. In E. Voorhees and D. Harman, editors , The Sixth Text REtrieval Conference (TREC-6) , pages 73 -- 82 , 1998 . NIST Special Publication 500-240. P. Over. Trec-6 interactive track report. In E. Voorhees and D. Harman, editors, The Sixth Text REtrieval Conference (TREC-6), pages 73--82, 1998. NIST Special Publication 500-240."},{"key":"e_1_2_1_10_1","first-page":"65","volume-title":"The Sixth Text REtrieval Conference (TREC-7)","author":"Over P.","year":"1999","unstructured":"P. Over . Trec-7 interactive track report. In E. Voorhees and D. Harman, editors , The Sixth Text REtrieval Conference (TREC-7) , pages 65 -- 72 , 1999 . NIST Special Publication 500-242. P. Over. Trec-7 interactive track report. In E. Voorhees and D. Harman, editors, The Sixth Text REtrieval Conference (TREC-7), pages 65--72, 1999. NIST Special Publication 500-242."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1108\/eb026647"},{"key":"e_1_2_1_12_1","first-page":"201","volume-title":"Proceedings of the 2nd Conference on Conceptions of Library and Information Science","author":"Saracevic T.","year":"1996","unstructured":"T. Saracevic . Relevance reconsidered . In Proceedings of the 2nd Conference on Conceptions of Library and Information Science , pages 201 -- 218 , 1996 . T. Saracevic. Relevance reconsidered. In Proceedings of the 2nd Conference on Conceptions of Library and Information Science, pages 201--218, 1996."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/331403.331404"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/502585.502654"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/383952.384019"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/564376.564393"}],"container-title":["ACM SIGIR Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2795403.2795405","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2795403.2795405","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T19:03:40Z","timestamp":1750273420000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2795403.2795405"}},"subtitle":["Methods and Evaluation Metrics for Subtopic Retrieval"],"short-title":[],"issued":{"date-parts":[[2015,6,23]]},"references-count":16,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2015,6,23]]}},"alternative-id":["10.1145\/2795403.2795405"],"URL":"https:\/\/doi.org\/10.1145\/2795403.2795405","relation":{},"ISSN":["0163-5840"],"issn-type":[{"value":"0163-5840","type":"print"}],"subject":[],"published":{"date-parts":[[2015,6,23]]},"assertion":[{"value":"2015-06-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}