{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:18:10Z","timestamp":1750306690038,"version":"3.41.0"},"reference-count":15,"publisher":"Association for Computing Machinery (ACM)","issue":"Spring","license":[{"start":{"date-parts":[[2014,4,1]],"date-time":"2014-04-01T00:00:00Z","timestamp":1396310400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001722","name":"Royal Netherlands Academy of Arts and Sciences","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001722","id-type":"DOI","asserted-by":"publisher"}]},{"name":"CLARIAH"},{"name":"Yahoo! Faculty Research and Engagement Program"},{"DOI":"10.13039\/100013407","name":"Netherlands eScience Center","doi-asserted-by":"crossref","award":["027.012.105"],"award-info":[{"award-number":["027.012.105"]}],"id":[{"id":"10.13039\/100013407","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004895","name":"European Social Fund","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004895","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004963","name":"Seventh Framework Programme","doi-asserted-by":"publisher","award":["288024"],"award-info":[{"award-number":["288024"]}],"id":[{"id":"10.13039\/501100004963","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003246","name":"Nederlandse Organisatie voor Wetenschappelijk Onderzoek","doi-asserted-by":"publisher","award":["727.011.005, 612.001.116, HOR-11-10"],"award-info":[{"award-number":["727.011.005, 612.001.116, HOR-11-10"]}],"id":[{"id":"10.13039\/501100003246","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Center for Creation, Content and Technology"},{"name":"Dutch national program COMMIT"},{"name":"CLARIN-nl"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGWEB Newsl."],"published-print":{"date-parts":[[2014,4]]},"abstract":"<jats:p>In this article we give an overview of our recent work on online learning to rank for information retrieval (IR). This work addresses IR from a reinforcement learning (RL) point of view, with the aim to enable systems that can learn directly from interactions with their users. Learning directly from user interactions is difficult for several reasons. First, user interactions are hard to interpret as feedback for learning because it is usually biased and noisy. Second, the system can only observe feedback on actions (e.g., rankers, documents) actually shown to users, which results in an exploration-exploitation challenge. Third, the amount of feedback and therefore the quality of learning is limited by the number of user interactions, so it is important to use the observed data as effectively as possible. Here, we discuss our work on interpreting user feedback using probabilistic interleaved comparisons, and on learning to rank from noisy, relative feedback.<\/jats:p>","DOI":"10.1145\/2591453.2591458","type":"journal-article","created":{"date-parts":[[2014,4,1]],"date-time":"2014-04-01T13:06:54Z","timestamp":1396357614000},"page":"1-7","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["\"Learning to rank for information retrieval from user interactions\" by K. Hofmann, S. Whiteson, A. Schuth, and M. de Rijke with Martin Vesely as coordinator"],"prefix":"10.1145","volume":"2014","author":[{"given":"Katja","family":"Hofmann","sequence":"first","affiliation":[{"name":"Microsoft Research"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shimon","family":"Whiteson","sequence":"additional","affiliation":[{"name":"Intelligent Systems Lab Amsterdam, University of Amsterdam"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anne","family":"Schuth","sequence":"additional","affiliation":[{"name":"Intelligent Systems Lab Amsterdam, University of Amsterdam"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maarten","family":"de Rijke","sequence":"additional","affiliation":[{"name":"Intelligent Systems Lab Amsterdam, University of Amsterdam"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2014,4]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2094072.2094078"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1645953.1646293"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2396761.2396780"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2433396.2433419"},{"key":"e_1_2_1_5_1","series-title":"Lecture Notes in Computer Science","volume-title":"ECIR '11","author":"HOFMANN K.","unstructured":"HOFMANN , K. , WHITESON , S. , AND DE RIJKE , M. 2011a. Balancing exploration and exploitation in learning to rank online . In ECIR '11 . Lecture Notes in Computer Science , vol. 6611 . Springer , 251--263. HOFMANN, K.,WHITESON, S., AND DE RIJKE, M. 2011a. Balancing exploration and exploitation in learning to rank online. In ECIR '11. Lecture Notes in Computer Science, vol. 6611. Springer, 251--263."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063576.2063618"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2396761.2398516"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-012-9197-9"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2536736.2536737"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775067"},{"key":"e_1_2_1_11_1","unstructured":"JOACHIMS T. 2003. Evaluating retrieval performance using clickthrough data. Text Mining.  JOACHIMS T. 2003. Evaluating retrieval performance using clickthrough data. Text Mining."},{"key":"e_1_2_1_12_1","doi-asserted-by":"crossref","unstructured":"RADLINSKI F. KURUP M. AND JOACHIMS T. 2008. How does clickthrough data reflect retrieval quality? In  RADLINSKI F. KURUP M. AND JOACHIMS T. 2008. How does clickthrough data reflect retrieval quality? In","DOI":"10.1145\/1458082.1458092"},{"volume-title":"'08","author":"CIKM","key":"e_1_2_1_13_1","unstructured":"CIKM '08 . ACM Press, 43--52. CIKM '08. ACM Press, 43--52."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2513150.2513162"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553527"}],"container-title":["ACM SIGWEB Newsletter"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2591453.2591458","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2591453.2591458","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T07:01:30Z","timestamp":1750230090000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2591453.2591458"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,4]]},"references-count":15,"journal-issue":{"issue":"Spring","published-print":{"date-parts":[[2014,4]]}},"alternative-id":["10.1145\/2591453.2591458"],"URL":"https:\/\/doi.org\/10.1145\/2591453.2591458","relation":{},"ISSN":["1931-1745","1931-1435"],"issn-type":[{"type":"print","value":"1931-1745"},{"type":"electronic","value":"1931-1435"}],"subject":[],"published":{"date-parts":[[2014,4]]},"assertion":[{"value":"2014-04-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}