{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T06:18:35Z","timestamp":1775283515847,"version":"3.50.1"},"reference-count":63,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2013,7,1]],"date-time":"2013-07-01T00:00:00Z","timestamp":1372636800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2013,7]]},"abstract":"<jats:p>Several questions remain unanswered by the existing literature concerning the deployment of query-dependent features within learning to rank. In this work, we investigate three research questions in order to empirically ascertain best practices for learning-to-rank deployments. (i) Previous work in data fusion that pre-dates learning to rank showed that while different retrieval systems could be effectively combined, the combination of multiple models within the same system was not as effective. In contrast, the existing learning-to-rank datasets (e.g., LETOR), often deploy multiple weighting models as query-dependent features within a single system, raising the question as to whether such a combination is needed. (ii) Next, we investigate whether the training of weighting model parameters, traditionally required for effective retrieval, is necessary within a learning-to-rank context. (iii) Finally, we note that existing learning-to-rank datasets use weighting model features calculated on different fields (e.g., title, content, or anchor text), even though such weighting models have been criticized in the literature. Experiments addressing these three questions are conducted on Web search datasets, using various weighting models as query-dependent and typical query-independent features, which are combined using three learning-to-rank techniques. In particular, we show and explain why multiple weighting models should be deployed as features. Moreover, we unexpectedly find that training the weighting model's parameters degrades learned model's effectiveness. Finally, we show that computing a weighting model separately for each field is less effective than more theoretically-sound field-based weighting models.<\/jats:p>","DOI":"10.1145\/2493175.2493176","type":"journal-article","created":{"date-parts":[[2013,8,1]],"date-time":"2013-08-01T15:14:00Z","timestamp":1375370040000},"page":"1-39","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":30,"title":["About learning models with multiple query-dependent features"],"prefix":"10.1145","volume":"31","author":[{"given":"Craig","family":"Macdonald","sequence":"first","affiliation":[{"name":"University of Glasgow, Glasgow, Scotland, U.K."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rodrygo L.T.","family":"Santos","sequence":"additional","affiliation":[{"name":"University of Glasgow, Glasgow, Scotland, U.K."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Iadh","family":"Ounis","sequence":"additional","affiliation":[{"name":"University of Glasgow, Glasgow, Scotland, U.K."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ben","family":"He","sequence":"additional","affiliation":[{"name":"University of Chinese Academy of Sciences"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2013,8,5]]},"reference":[{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the 16th Text REtrieval Conference (TREC'07)","author":"Amati G.","unstructured":"Amati , G. , Ambrosi , E. , Bianchi , M. , Gaibisso , C. , and Gambosi , G . 2008. FUB, IASI-CNR and University of Tor Vergata at TREC 2007 Blog track . In Proceedings of the 16th Text REtrieval Conference (TREC'07) . Amati, G., Ambrosi, E., Bianchi, M., Gaibisso, C., and Gambosi, G. 2008. FUB, IASI-CNR and University of Tor Vergata at TREC 2007 Blog track. In Proceedings of the 16th Text REtrieval Conference (TREC'07)."},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the 2nd International Workshop on Adversarial Information Retrieval on the Web (AIRWEB'06)","author":"Becchetti L.","unstructured":"Becchetti , L. , Castillo , C. , Donato , D. , Leonardi , S. , and Baeza-Yates , R . 2006. Link-based characterization and detection of Web spam . In Proceedings of the 2nd International Workshop on Adversarial Information Retrieval on the Web (AIRWEB'06) . Becchetti, L., Castillo, C., Donato, D., Leonardi, S., and Baeza-Yates, R. 2006. Link-based characterization and detection of Web spam. In Proceedings of the 2nd International Workshop on Adversarial Information Retrieval on the Web (AIRWEB'06)."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.20012"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935849"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/956863.956944"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102363"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 18th Text REtrieval Conference (TREC'09)","author":"Callan J.","unstructured":"Callan , J. , Hoy , M. , Yoo , C. , and Zhao , L . 2009. The ClueWeb09 dataset . In Proceedings of the 18th Text REtrieval Conference (TREC'09) . Callan, J., Hoy, M., Yoo, C., and Zhao, L. 2009. The ClueWeb09 dataset. In Proceedings of the 18th Text REtrieval Conference (TREC'09)."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1718487.1718538"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273496.1273513"},{"key":"e_1_2_1_11_1","first-page":"1","article-title":"Yahoo&excl; learning to rank challenge overview","volume":"14","author":"Chappelle O.","year":"2011","unstructured":"Chappelle , O. and Chang , Y. 2011 . Yahoo&excl; learning to rank challenge overview . J. Machine Learn. Res.-Proc. Track , 14 , 1 -- 24 . Chappelle, O. and Chang, Y. 2011. Yahoo&excl; learning to rank challenge overview. J. Machine Learn. Res.-Proc. Track, 14, 1--24.","journal-title":"J. Machine Learn. Res.-Proc. Track"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/564376.564454"},{"key":"e_1_2_1_13_1","volume-title":"Overview of the TREC 2009 Web track. In Proceedings of the 18th Text REtrieval Conference (TREC'09)","author":"Clarke C. L. A.","unstructured":"Clarke , C. L. A. , Craswell , N. , and Soboroff , I . 2010 . Overview of the TREC 2009 Web track. In Proceedings of the 18th Text REtrieval Conference (TREC'09) . Clarke, C. L. A., Craswell, N., and Soboroff, I. 2010. Overview of the TREC 2009 Web track. In Proceedings of the 18th Text REtrieval Conference (TREC'09)."},{"key":"e_1_2_1_14_1","volume-title":"Overview of the TREC 2010 Web track. In Proceedings of the 19th Text REtrieval Conference (TREC'10)","author":"Clarke C. L. A.","unstructured":"Clarke , C. L. A. , Craswell , N. , and Soboroff , I . 2011 . Overview of the TREC 2010 Web track. In Proceedings of the 19th Text REtrieval Conference (TREC'10) . Clarke, C. L. A., Craswell, N., and Soboroff, I. 2011. Overview of the TREC 2010 Web track. In Proceedings of the 19th Text REtrieval Conference (TREC'10)."},{"key":"e_1_2_1_15_1","volume-title":"Overview of the TREC 2011 Web track. In Proceedings of the 20th Text REtrieval Conference (TREC'11)","author":"Clarke C. L. A.","unstructured":"Clarke , C. L. A. , Craswell , N. , and Soboroff , I . 2012 . Overview of the TREC 2011 Web track. In Proceedings of the 20th Text REtrieval Conference (TREC'11) . Clarke, C. L. A., Craswell, N., and Soboroff, I. 2012. Overview of the TREC 2011 Web track. In Proceedings of the 20th Text REtrieval Conference (TREC'11)."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-011-9162-z"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 18th Text REtrieval Conference (TREC'09)","author":"Craswell N.","unstructured":"Craswell , N. , Fetterly , D. , Najork , M. , Robertson , S. , and Yilmaz , E . 2010. Microsoft Research at TREC 2009 . In Proceedings of the 18th Text REtrieval Conference (TREC'09) . Craswell, N., Fetterly, D., Najork, M., Robertson, S., and Yilmaz, E. 2010. Microsoft Research at TREC 2009. In Proceedings of the 18th Text REtrieval Conference (TREC'09)."},{"key":"e_1_2_1_18_1","volume-title":"Search Engines: Information Retrieval in Practice","author":"Croft B.","year":"2009","unstructured":"Croft , B. , Metzler , D. , and Strohman , T . 2009 . Search Engines: Information Retrieval in Practice . Addison-Wesley , Boston, MA . Croft, B., Metzler, D., and Strohman, T. 2009. Search Engines: Information Retrieval in Practice. Addison-Wesley, Boston, MA."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1008992.1009004"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the 1st Text REtrieval Conference (TREC-1). 319--328","author":"Fox E. A.","unstructured":"Fox , E. A. , Koushik , P. , Shaw , J. A. , Modlin , R. , and Rao , D . 1993. Combining evidence from multiple searches . In Proceedings of the 1st Text REtrieval Conference (TREC-1). 319--328 . NIST Special Publication 500--207, Gaithersburg, MD. Fox, E. A., Koushik, P., Shaw, J. A., Modlin, R., and Rao, D. 1993. Combining evidence from multiple searches. In Proceedings of the 1st Text REtrieval Conference (TREC-1). 319--328. NIST Special Publication 500--207, Gaithersburg, MD."},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the 2nd Text REtrieval Conference (TREC-2). 243--252","author":"Fox E. A.","unstructured":"Fox , E. A. and Shaw , J. A . 1994. Combination of multiple searches . In Proceedings of the 2nd Text REtrieval Conference (TREC-2). 243--252 . NIST Special Publication 500--215, Gaithersburg, MD. Fox, E. A. and Shaw, J. A. 1994. Combination of multiple searches. In Proceedings of the 2nd Text REtrieval Conference (TREC-2). 243--252. NIST Special Publication 500--215, Gaithersburg, MD."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2009932"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390334.1390348"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/956863.956867"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076114"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2006.11.002"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1247715.1247719"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775067"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(00)00015-7"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the 14th Text REtrieval Conference (TREC'05)","author":"Kamps J.","year":"2006","unstructured":"Kamps , J. 2006 . Effective smoothing for a terabyte of text . In Proceedings of the 14th Text REtrieval Conference (TREC'05) . NIST Special Publication, Gaithersburg, MD. Kamps, J. 2006. Effective smoothing for a terabyte of text. In Proceedings of the 14th Text REtrieval Conference (TREC'05). NIST Special Publication, Gaithersburg, MD."},{"key":"e_1_2_1_32_1","first-page":"261","article-title":"A study of the overlap among document representations","volume":"1","author":"Katzer J.","year":"1982","unstructured":"Katzer , J. , McGill , M. J. , Tessier , J. , Frakes , W. , and Dasgupta , P. 1982 . A study of the overlap among document representations . Inf. Technol. Res. Develo. 1 , 2, 261 -- 274 . Katzer, J., McGill, M. J., Tessier, J., Frakes, W., and Dasgupta, P. 1982. A study of the overlap among document representations. Inf. Technol. Res. Develo. 1, 2, 261--274.","journal-title":"Inf. Technol. Res. Develo."},{"key":"e_1_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Kirkpatrick S. Gelatt C. D. and Vecchi M. P. 1983. Optimization by simulated annealing. Science 220 4598 671--680.  Kirkpatrick S. Gelatt C. D. and Vecchi M. P. 1983. Optimization by simulated annealing. Science 220 4598 671--680.","DOI":"10.1126\/science.220.4598.671"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/564376.564383"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/258525.258587"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/382979.383041"},{"key":"e_1_2_1_37_1","volume-title":"Learning to Rank for Information Retrieval and Natural Language Processing. Synthesis Lectures on Human Language Technologies","author":"Li H.","unstructured":"Li , H. 2011. Learning to Rank for Information Retrieval and Natural Language Processing. Synthesis Lectures on Human Language Technologies , Morgan & Claypool Publishers . Li, H. 2011. Learning to Rank for Information Retrieval and Natural Language Processing. Synthesis Lectures on Human Language Technologies, Morgan & Claypool Publishers."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000016"},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the SIGIR Workshop in Open Source Information Retrieval.","author":"Macdonald C.","unstructured":"Macdonald , C. , McCreadie , R. , Santos , R. , and Ounis , I . 2012. From puppy to maturity: Experiences in developing terrier . In Proceedings of the SIGIR Workshop in Open Source Information Retrieval. Macdonald, C., McCreadie, R., Santos, R., and Ounis, I. 2012. From puppy to maturity: Experiences in developing terrier. In Proceedings of the SIGIR Workshop in Open Source Information Retrieval."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/11878773_100"},{"key":"e_1_2_1_41_1","unstructured":"Macdonald C. Santos R. and Ounis I. 2012. The whens and hows of learning to rank for web search. Info. Retrieval 10.1007\/s10791-012-9209-9.  Macdonald C. Santos R. and Ounis I. 2012. The whens and hows of learning to rank for web search. Info. Retrieval 10.1007\/s10791-012-9209-9."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277859"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1321440.1321478"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/237496.237497"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/860435.860463"},{"key":"e_1_2_1_46_1","unstructured":"Page L. Brin S. Motwani R. and Winograd T. 1998. The PageRank citation ranking: Bringing order to the Web. Tech. rep. Stanford Digital Library Technologies Project Stanford CA.  Page L. Brin S. Motwani R. and Winograd T. 1998. The PageRank citation ranking: Bringing order to the Web. Tech. rep. Stanford Digital Library Technologies Project Stanford CA."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277937"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.5555\/2011166.2011171"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-009-9123-y"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/1031171.1031181"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.4630270302"},{"key":"e_1_2_1_53_1","volume-title":"Proceedings of the 4th Text REtrieval Conference (TREC-4).","author":"Robertson S. E.","unstructured":"Robertson , S. E. , Walker , S. , Hancock-Beaulieu , M. , Gatford , M. , and Payne , A . 1995. Okapi at TREC-4 . In Proceedings of the 4th Text REtrieval Conference (TREC-4). Robertson, S. E., Walker, S., Hancock-Beaulieu, M., Gatford, M., and Payne, A. 1995. Okapi at TREC-4. In Proceedings of the 4th Text REtrieval Conference (TREC-4)."},{"key":"e_1_2_1_54_1","volume-title":"Proceedings of the Text REtrieval Conference (TREC-1). 21--30","author":"Robertson S. E.","unstructured":"Robertson , S. E. , Walker , S. , Hancock-Beaulieu , M. , Gull , A. , and Lau , M . 1992. Okapi at TREC . In Proceedings of the Text REtrieval Conference (TREC-1). 21--30 . Robertson, S. E., Walker, S., Hancock-Beaulieu, M., Gull, A., and Lau, M. 1992. Okapi at TREC. In Proceedings of the Text REtrieval Conference (TREC-1). 21--30."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/243199.243206"},{"key":"e_1_2_1_56_1","unstructured":"Smucker M. D. and Allan J. 2005. An investigation of dirichlet prior smoothing's performance advantage. Tech. rep. IR-391 University of Massachusetts Amherst MA.  Smucker M. D. and Allan J. 2005. An investigation of dirichlet prior smoothing's performance advantage. Tech. rep. IR-391 University of Massachusetts Amherst MA."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(95)00020-H"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/1963405.1963461"},{"key":"e_1_2_1_59_1","volume-title":"Information Retrieval","author":"van Rijsbergen C.","unstructured":"van Rijsbergen , C. 1979. Information Retrieval , 2 nd Ed. Butterworths, London . U.K. van Rijsbergen, C. 1979. Information Retrieval, 2nd Ed. Butterworths, London. U.K.","edition":"2"},{"key":"e_1_2_1_60_1","volume-title":"Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'97)","author":"Vogt C.","year":"1997","unstructured":"Vogt , C. 1997 . When does it make sense to linearly combine relevance scores . In Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'97) . ACM, New York, NY. Vogt, C. 1997. When does it make sense to linearly combine relevance scores. In Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'97). ACM, New York, NY."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.290991"},{"key":"e_1_2_1_62_1","first-page":"1","article-title":"Tree ensembles and transfer learning","volume":"14","author":"Weinberger K.","year":"2010","unstructured":"Weinberger , K. , Mohan , A. , and Chen , Z. 2010 . Tree ensembles and transfer learning . In J. Machine Learn. Res.-Proc. Track. 14 , 1 -- 24 . Weinberger, K., Mohan, A., and Chen, Z. 2010. Tree ensembles and transfer learning. In J. Machine Learn. Res.-Proc. Track. 14, 1--24.","journal-title":"J. Machine Learn. Res.-Proc. Track."},{"key":"e_1_2_1_63_1","unstructured":"Wu Q. Burges C. J. C. Svore K. M. and Gao J. 2008. Ranking boosting and model adaptation. Tech. rep. MSR-TR-2008-109 Microsoft Redmand WA.  Wu Q. Burges C. J. C. Svore K. M. and Gao J. 2008. Ranking boosting and model adaptation. Tech. rep. MSR-TR-2008-109 Microsoft Redmand WA."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277809"},{"key":"e_1_2_1_65_1","volume-title":"Proceedings of the 13th Text REtrieval Conference (TREC'04)","author":"Zaragoza H.","unstructured":"Zaragoza , H. , Craswell , N. , Taylor , M. , Saria , S. , and Robertson , S . 2004. Microsoft Cambridge at TREC-13: Web and HARD tracks . In Proceedings of the 13th Text REtrieval Conference (TREC'04) . Zaragoza, H., Craswell, N., Taylor, M., Saria, S., and Robertson, S. 2004. Microsoft Cambridge at TREC-13: Web and HARD tracks. In Proceedings of the 13th Text REtrieval Conference (TREC'04)."},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/383952.384019"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2493175.2493176","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2493175.2493176","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T08:39:33Z","timestamp":1750235973000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2493175.2493176"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,7]]},"references-count":63,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2013,7]]}},"alternative-id":["10.1145\/2493175.2493176"],"URL":"https:\/\/doi.org\/10.1145\/2493175.2493176","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2013,7]]},"assertion":[{"value":"2012-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-02-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-08-05","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}