{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T03:12:00Z","timestamp":1761621120148,"version":"3.41.0"},"reference-count":37,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2012,11,1]],"date-time":"2012-11-01T00:00:00Z","timestamp":1351728000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2012,11]]},"abstract":"<jats:p>Traditional machine-learned ranking systems for Web search are often trained to capture stationary relevance of documents to queries, which have limited ability to track nonstationary user intention in a timely manner. In recency search, for instance, the relevance of documents to a query on breaking news often changes significantly over time, requiring effective adaptation to user intention. In this article, we focus on recency search and study a number of algorithms to improve ranking results by leveraging user click feedback. Our contributions are threefold. First, we use commercial search engine sessions collected in a random exploration bucket for reliable offline evaluation of these algorithms, which provides an unbiased comparison across algorithms without online bucket tests. Second, we propose an online learning approach that reranks and improves the search results for recency queries near real-time based on user clicks. This approach is very general and can be combined with sophisticated click models. Third, our empirical comparison of a dozen algorithms on real-world search data suggests importance of a few algorithmic choices in these applications, including generalization across different query-document pairs, specialization to popular queries, and near real-time adaptation of user clicks for reranking.<\/jats:p>","DOI":"10.1145\/2382438.2382439","type":"journal-article","created":{"date-parts":[[2012,11,29]],"date-time":"2012-11-29T15:02:27Z","timestamp":1354201347000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["An Online Learning Framework for Refining Recency Search Results with User Click Feedback"],"prefix":"10.1145","volume":"30","author":[{"given":"Taesup","family":"Moon","sequence":"first","affiliation":[{"name":"Yahoo! Labs"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wei","family":"Chu","sequence":"additional","affiliation":[{"name":"Yahoo! Labs"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lihong","family":"Li","sequence":"additional","affiliation":[{"name":"Yahoo! Labs"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhaohui","family":"Zheng","sequence":"additional","affiliation":[{"name":"Yahoo! Labs"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yi","family":"Chang","sequence":"additional","affiliation":[{"name":"Yahoo! Labs"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2012,11]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2009.52"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835804.1835894"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102363"},{"key":"e_1_2_1_4_1","volume-title":"Advances in Neural Information Processing Systems","volume":"19","author":"Burges C. J.","unstructured":"Burges , C. J. , Le , Q. V. , and Ragno , R . 2007. Learning to rank with nonsmooth cost functions . In Advances in Neural Information Processing Systems , vol. 19 , B. Sch\u00f6lkopf, J. Platt, and T. Hofmann Eds. Burges, C. J., Le, Q. V., and Ragno, R. 2007. Learning to rank with nonsmooth cost functions. In Advances in Neural Information Processing Systems, vol. 19, B. Sch\u00f6lkopf, J. Platt, and T. Hofmann Eds."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1526709.1526711"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273496.1273518"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835449.1835471"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1718487.1718490"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772725"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1718487.1718510"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1718487.1718489"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/945365.964285"},{"volume-title":"Proceedings of the 27th International Conference on Machine Learning. 13--20","author":"Graepel T.","key":"e_1_2_1_13_1","unstructured":"Graepel , T. , Candela , J. Q. , Borchert , T. , and Herbrich , R . 2010. Web-scale Bayesian click-through rate prediction for sponsored search advertising in Microsoft\u2019s Bing search engine . In Proceedings of the 27th International Conference on Machine Learning. 13--20 . Graepel, T., Candela, J. Q., Borchert, T., and Herbrich, R. 2010. Web-scale Bayesian click-through rate prediction for sponsored search advertising in Microsoft\u2019s Bing search engine. In Proceedings of the 27th International Conference on Machine Learning. 13--20."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1526709.1526712"},{"volume-title":"Proceedings of the 24th AAAI Conference on Artificial Intelligence.","author":"Inagaki Y.","key":"e_1_2_1_15_1","unstructured":"Inagaki , Y. , Sadagopan , N. , Dupret , G. , Liao , C. , Dong , A. , Chang , Y. , and Zheng , Z . 2010. Session based click features for recency ranking . In Proceedings of the 24th AAAI Conference on Artificial Intelligence. Inagaki, Y., Sadagopan, N., Dupret, G., Liao, C., Dong, A., Chang, Y., and Zheng, Z. 2010. Session based click features for recency ranking. In Proceedings of the 24th AAAI Conference on Artificial Intelligence."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1571941.1571950"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775067"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775067"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1247715.1247720"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935924"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1557019.1557072"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935862"},{"key":"e_1_2_1_24_1","unstructured":"Langford J. Li L. and Strehl A. 2007. Vowpal wabbit online learning project. http:\/\/hunch.net\/?p=309. Langford J. Li L. and Strehl A. 2007. Vowpal wabbit online learning project. http:\/\/hunch.net\/?p=309."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772758"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935878"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1645953.1646035"},{"volume-title":"Learning to Rank for Information Retrieval","author":"Liu T. Y.","key":"e_1_2_1_28_1","unstructured":"Liu , T. Y. 2009. Learning to Rank for Information Retrieval . Now Publishers . Liu, T. Y. 2009. Learning to Rank for Information Retrieval. Now Publishers."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1871437.1871657"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1281192.1281254"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458092"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1090\/S0002-9904-1952-09620-8"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1526709.1526725"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553527"},{"volume-title":"Query Log Analysis: Social and Technological Challenges. A Workshop at the 16th International World Wide Web Conference.","author":"Zhang V.","key":"e_1_2_1_35_1","unstructured":"Zhang , V. and Jones , R . 2007. Comparing click logs and editorial labels for training query rewriting . In Query Log Analysis: Social and Technological Challenges. A Workshop at the 16th International World Wide Web Conference. Zhang, V. and Jones, R. 2007. Comparing click logs and editorial labels for training query rewriting. In Query Log Analysis: Social and Technological Challenges. A Workshop at the 16th International World Wide Web Conference."},{"key":"e_1_2_1_36_1","volume-title":"Advances in Neural Information Processing Systems","volume":"20","author":"Zheng Z.","unstructured":"Zheng , Z. , Zha , H. , Zhang , T. , Chapelle , O. , and Chen , K . 2008. A general boosting method and its application to learning ranking functions for web search . In Advances in Neural Information Processing Systems , vol. 20 . MIT Press, Cambridge, MA. Zheng, Z., Zha, H., Zhang, T., Chapelle, O., and Chen, K. 2008. A general boosting method and its application to learning ranking functions for web search. In Advances in Neural Information Processing Systems, vol. 20. MIT Press, Cambridge, MA."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1718487.1718528"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2382438.2382439","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2382438.2382439","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T09:34:49Z","timestamp":1750239289000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2382438.2382439"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,11]]},"references-count":37,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2012,11]]}},"alternative-id":["10.1145\/2382438.2382439"],"URL":"https:\/\/doi.org\/10.1145\/2382438.2382439","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"type":"print","value":"1046-8188"},{"type":"electronic","value":"1558-2868"}],"subject":[],"published":{"date-parts":[[2012,11]]},"assertion":[{"value":"2011-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-11-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}