{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T10:39:43Z","timestamp":1753439983972,"version":"3.41.0"},"reference-count":94,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2021,12,8]],"date-time":"2021-12-08T00:00:00Z","timestamp":1638921600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"HK PolyU","award":["P0030932"],"award-info":[{"award-number":["P0030932"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2022,7,31]]},"abstract":"<jats:p>In Information Retrieval, numerous retrieval models or document ranking functions have been developed in the quest for better retrieval effectiveness. Apart from some formal retrieval models formulated on a theoretical basis, various recent works have applied heuristic constraints to guide the derivation of document ranking functions. While many recent methods are shown to improve over established and successful models, comparison among these new methods under a common environment is often missing. To address this issue, we perform an extensive and up-to-date comparison of leading term-independence retrieval models implemented in our own retrieval system. Our study focuses on the following questions: (RQ1) Is there a retrieval model that consistently outperforms all other models across multiple collections; (RQ2) What are the important features of an effective document ranking function? Our retrieval experiments performed on several TREC test collections of a wide range of sizes (up to the terabyte-sized Clueweb09 Category B) enable us to answer these research questions. This work also serves as a reproducibility study for leading retrieval models. While our experiments show that no single retrieval model outperforms all others across all tested collections, some recent retrieval models, such as MATF and MVD, consistently perform better than the common baselines.<\/jats:p>","DOI":"10.1145\/3483612","type":"journal-article","created":{"date-parts":[[2021,12,8]],"date-time":"2021-12-08T20:03:55Z","timestamp":1638993835000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["A Comparison between Term-Independence Retrieval Models for Ad Hoc Retrieval"],"prefix":"10.1145","volume":"40","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4917-7216","authenticated-orcid":false,"given":"Edward Kai Fung","family":"Dang","sequence":"first","affiliation":[{"name":"The Hong Kong Polytechnic University, Hung Hom, Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert Wing Pong","family":"Luk","sequence":"additional","affiliation":[{"name":"The Hong Kong Polytechnic University, Hung Hom, Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James","family":"Allan","sequence":"additional","affiliation":[{"name":"University of Massachusetts, Amherst, MA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,12,8]]},"reference":[{"key":"e_1_3_3_2_2","first-page":"310","volume-title":"Proceedings of the Workshop of the Cross-Language Evaluation Forum for European Languages","author":"Amati G.","year":"2003","unstructured":"G. Amati, C. Carpineto, and G. Romano. 2003. Comparing weighting models for monolingual information retrieval. In Proceedings of the Workshop of the Cross-Language Evaluation Forum for European Languages. 310\u2013318."},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582416"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/1645953.1646031"},{"key":"e_1_3_3_5_2","first-page":"125","volume-title":"Proceedings of the 1st International Conference on Theory of Information Retrieval","author":"Azzopardi L.","year":"2007","unstructured":"L. Azzopardi and T. Roelleke. 2007. Explicitly considering relevance within the language modeling framework. In Proceedings of the 1st International Conference on Theory of Information Retrieval. 125\u2013134."},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1995.tb02031.x"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.5555\/1378307.1378323"},{"key":"e_1_3_3_8_2","volume-title":"Proceedings of the 2006 Text Retrieval Conference","author":"B\u00fcttcher S.","year":"2006","unstructured":"S. B\u00fcttcher, C. L. A. Clarke, and I. Soboroff. 2006. The TREC 2006 terabyte track. In Proceedings of the 2006 Text Retrieval Conference."},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/2094072.2094076"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/1194936.1194941"},{"key":"e_1_3_3_11_2","volume-title":"Proceedings of the 21st Text REtreival Conference","author":"Clarke Charles L. A.","year":"2012","unstructured":"Charles L. A. Clarke, Nick Craswell, and Ellen M. Voorhees. 2012. Overview of the TREC 2012 web track. In Proceedings of the 21st Text REtreival Conference."},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/1835449.1835490"},{"key":"e_1_3_3_13_2","first-page":"100","volume-title":"Proceedings of the Workshop on Continuous Vector Space Models and Their Compositionality","author":"Clinchant St\u00e9phane","year":"2013","unstructured":"St\u00e9phane Clinchant and Florent Perronnin. 2013. Aggregating continuous word embeddings for information retrieval. In Proceedings of the Workshop on Continuous Vector Space Models and Their Compositionality. 100\u2013109."},{"key":"e_1_3_3_14_2","volume-title":"Proceedings of the TREC-2 Conference","author":"Cooper W. S.","year":"1994","unstructured":"W. S. Cooper, A. Chen, and F. C. Gey. 1994. Full text retrieval based on probabilistic equations with coefficients fitted by logistic regression. In Proceedings of the TREC-2 Conference."},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/1571941.1572114"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-011-9162-z"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/2746231"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/0-387-29295-0_78"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.5555\/1890706.1890716"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/1629096.1629098"},{"key":"e_1_3_3_21_2","volume-title":"Proceedings of the 21st Text REtreival Conference","author":"Din\u00e7er B. T.","year":"2012","unstructured":"B. T. Din\u00e7er. 2012. IRRA at TREC 2012: Divergence from independence (DFI). In Proceedings of the 21st Text REtreival Conference."},{"key":"e_1_3_3_22_2","volume-title":"Proceedings of the 19th Text REtreival Conference","author":"Din\u00e7er B. T.","year":"2010","unstructured":"B. T. Din\u00e7er, \u0130 Kocaba\u015f, and B. Karaog\u0303lan. 2010. IRRA at TREC 2010: Index term weighting by divergence from independence model. In Proceedings of the 19th Text REtreival Conference."},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00074"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/1008992.1009004"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/1961209.1961210"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076116"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3190580.3190586"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/2766462.2767780"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2016.11.003"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/2983323.2983769"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2019.102067"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-15712-8_41"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/956863.956867"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076114"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/1321440.1321479"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.5555\/646631.699450"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/2661829.2661894"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210147"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2914748"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1111\/coin.12248"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-45442-5_4"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCAE.2010.5451939"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/2882782"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-013-9225-4"},{"key":"e_1_3_3_45_2","volume-title":"Proceedings of the ACM SIGIR Workshop on Mathematical\/Formal Methods for Information Retrieval","author":"Kong Y. K.","year":"2004","unstructured":"Y. K. Kong, Robert W. P. Luk, W. Lam, K. S. Ho, and F. L. Chung. 2004. Passage-based retrieval using parameterized fuzzy set operators. In Proceedings of the ACM SIGIR Workshop on Mathematical\/Formal Methods for Information Retrieval."},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/203052.203067"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/383952.383970"},{"key":"e_1_3_3_48_2","first-page":"1","volume-title":"Proceedings of the Language Modeling for Information Retrieval","author":"Lafferty J.","year":"2001","unstructured":"J. Lafferty and C. X. Zhai. 2001. Probabilistic relevance models based on document and query generation. In Proceedings of the Language Modeling for Information Retrieval. Dordrecht: Springer, 1\u201310."},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-017-0171-6_1"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/383952.383972"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/1571941.1572023"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/52.582976"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277914"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.5555\/249622.249628"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/772755.772758"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-008-9062-z"},{"key":"e_1_3_3_57_2","article-title":"Why is information retrieval a scientific discipline","author":"Luk Robert W. P.","year":"2021","unstructured":"Robert W. P. Luk. 2021. Why is information retrieval a scientific discipline. Foundations of Science (2021), 1\u201327. DOI:https:\/\/doi.org\/10.1007\/s10699-020-09685-x","journal-title":"Foundations of Science"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/1645953.1646259"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/2063576.2063584"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076115"},{"key":"e_1_3_3_61_2","unstructured":"Bhaskar Mitra Eric Nalisnick Nick Craswell and Rich Caruana. 2016. A dual embedding space model for document ranking. arXiv:1602.01137v1. https:\/\/arxiv.org\/abs\/1602.01137v1"},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/584792.584881"},{"key":"e_1_3_3_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/2699669"},{"key":"e_1_3_3_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3511210"},{"key":"e_1_3_3_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484070"},{"key":"e_1_3_3_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/2766462.2767762"},{"key":"e_1_3_3_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2011.63"},{"key":"e_1_3_3_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/290941.291008"},{"key":"e_1_3_3_69_2","doi-asserted-by":"publisher","DOI":"10.1108\/eb046814"},{"key":"e_1_3_3_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/3360487"},{"key":"e_1_3_3_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2911508"},{"key":"e_1_3_3_72_2","volume-title":"Proceedings of the 7th Text REtreival Conference","author":"Robertson S. E.","year":"1999","unstructured":"S. E. Robertson, S. Walker, and M. Beaulieu. 1999. Okapi at TREC-7: Automatic ad hoc, filtering, VLC and interactive track. In Proceedings of the 7th Text REtreival Conference."},{"key":"e_1_3_3_73_2","volume-title":"Proceedings of the 3rd Text REtreival Conference","author":"Robertson S. E.","year":"1995","unstructured":"S. E. Robertson, S. Walker, S. Jones, M. M. Hancock-Beauleiu, and M. Gatford. 1995. Okapi at TREC-3. In Proceedings of the 3rd Text REtreival Conference."},{"key":"e_1_3_3_74_2","doi-asserted-by":"publisher","DOI":"10.1561\/1500000019"},{"key":"e_1_3_3_75_2","doi-asserted-by":"publisher","DOI":"10.1108\/eb026647"},{"key":"e_1_3_3_76_2","doi-asserted-by":"publisher","DOI":"10.5555\/188490.188561"},{"key":"e_1_3_3_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331357"},{"key":"e_1_3_3_78_2","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(88)90021-0"},{"key":"e_1_3_3_79_2","doi-asserted-by":"publisher","DOI":"10.1145\/1105696.1105701"},{"key":"e_1_3_3_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/243199.243206"},{"key":"e_1_3_3_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/1321440.1321528"},{"key":"e_1_3_3_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/2.989940"},{"key":"e_1_3_3_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148201"},{"key":"e_1_3_3_84_2","doi-asserted-by":"publisher","DOI":"10.1145\/2682862.2682863"},{"key":"e_1_3_3_85_2","doi-asserted-by":"publisher","DOI":"10.1093\/comjnl\/35.3.279"},{"key":"e_1_3_3_86_2","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(79)90030-X"},{"key":"e_1_3_3_87_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2006.10.009"},{"key":"e_1_3_3_88_2","doi-asserted-by":"publisher","DOI":"10.1145\/2808194.2809446"},{"key":"e_1_3_3_89_2","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080809"},{"key":"e_1_3_3_90_2","doi-asserted-by":"publisher","DOI":"10.1145\/1571941.1571954"},{"key":"e_1_3_3_91_2","doi-asserted-by":"publisher","DOI":"10.1145\/2970398.2970415"},{"key":"e_1_3_3_92_2","doi-asserted-by":"publisher","DOI":"10.1145\/3239571"},{"key":"e_1_3_3_93_2","doi-asserted-by":"publisher","DOI":"10.1145\/502585.502654"},{"key":"e_1_3_3_94_2","doi-asserted-by":"publisher","DOI":"10.1145\/564376.564387"},{"key":"e_1_3_3_95_2","doi-asserted-by":"publisher","DOI":"10.1145\/984321.984322"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3483612","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3483612","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:19:03Z","timestamp":1750191543000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3483612"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,8]]},"references-count":94,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,7,31]]}},"alternative-id":["10.1145\/3483612"],"URL":"https:\/\/doi.org\/10.1145\/3483612","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"type":"print","value":"1046-8188"},{"type":"electronic","value":"1558-2868"}],"subject":[],"published":{"date-parts":[[2021,12,8]]},"assertion":[{"value":"2020-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-12-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}