{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,13]],"date-time":"2026-05-13T13:42:41Z","timestamp":1778679761444,"version":"3.51.4"},"reference-count":77,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2014,10,28]],"date-time":"2014-10-28T00:00:00Z","timestamp":1414454400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2014,10,28]]},"abstract":"<jats:p>\n            Current research on Web search has focused on optimizing and evaluating single queries. However, a significant fraction of user queries are part of more complex tasks [Jones and Klinkner 2008] which span multiple queries across one or more search sessions [Liu and Belkin 2010; Kotov et al. 2011]. An ideal search engine would not only retrieve relevant results for a user's particular query but also be able to identify when the user is engaged in a more complex task and aid the user in completing that task [Morris et al. 2008; Agichtein et al. 2012]. Toward optimizing whole-session or task relevance, we characterize and address the problem of\n            <jats:italic>intrinsic diversity<\/jats:italic>\n            (ID) in retrieval [Radlinski et al. 2009], a type of complex task that requires multiple interactions with current search engines. Unlike existing work on extrinsic diversity [Carbonell and Goldstein 1998; Zhai et al. 2003; Chen and Karger 2006] that deals with ambiguity in intent across multiple users, ID queries often have little ambiguity in intent but seek content covering a variety of aspects on a shared theme. In such scenarios, the underlying needs are typically exploratory, comparative, or breadth-oriented in nature. We identify and address three key problems for ID retrieval: identifying authentic examples of ID tasks from post-hoc analysis of behavioral signals in search logs; learning to identify initiator queries that mark the start of an ID search task; and given an initiator query, predicting which content to prefetch and rank.\n          <\/jats:p>","DOI":"10.1145\/2629553","type":"journal-article","created":{"date-parts":[[2014,10,29]],"date-time":"2014-10-29T18:05:55Z","timestamp":1414605955000},"page":"1-45","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Understanding Intrinsic Diversity in Web Search"],"prefix":"10.1145","volume":"32","author":[{"given":"Karthik","family":"Raman","sequence":"first","affiliation":[{"name":"Microsoft Research"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paul N.","family":"Bennett","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kevyn","family":"Collins-Thompson","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2014,10,28]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2348283.2348328"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1498759.1498766"},{"key":"e_1_2_1_3_1","unstructured":"Peter Bailey Liwei Chen Scott Grosenick Li Jiang Yan Li Paul Reinholdtsen Charles Salada Haidong Wang and Sandy Wong. 2012. User task understanding: A web search engine perspective. http:\/\/research.microsoft.com\/apps\/-pubs\/default.aspx&quest;id=180594.  Peter Bailey Liwei Chen Scott Grosenick Li Jiang Yan Li Paul Reinholdtsen Charles Salada Haidong Wang and Sandy Wong. 2012. User task understanding: A web search engine perspective. http:\/\/research.microsoft.com\/apps\/-pubs\/default.aspx&quest;id=180594."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835449.1835606"},{"key":"e_1_2_1_5_1","volume-title":"Mcauliffe","author":"Bartlett Peter L.","year":"2004","unstructured":"Peter L. Bartlett , Michael I. Jordan , and Jon M . Mcauliffe . 2004 . Large margin classifiers: Convex loss, low noise, and convergence rates. In Advances in Neural Information Processing Systems 16. MIT Press , 1173--1180. DOI: http:\/\/papers.nips.cc\/paper\/2416-large-margin-classifiers-convex-loss-low- noise-and-convergence-rates.pdf. Peter L. Bartlett, Michael I. Jordan, and Jon M. Mcauliffe. 2004. Large margin classifiers: Convex loss, low noise, and convergence rates. In Advances in Neural Information Processing Systems 16. MIT Press, 1173--1180. DOI: http:\/\/papers.nips.cc\/paper\/2416-large-margin-classifiers-convex-loss-low- noise-and-convergence-rates.pdf."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772703"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935872"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1137\/080733991"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.14778\/1988776.1988781"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.291025"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148245"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390334.1390446"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04417-5_17"},{"key":"e_1_2_1_14_1","volume-title":"Proceedings of the SIGIR Workshop on Faceted Search.","author":"Dakka Wisam","unstructured":"Wisam Dakka , Rishabh Dayal , and Panagiotis G. Ipeirotis . 2006. Automatic discovery of useful facet terms . In Proceedings of the SIGIR Workshop on Faceted Search. Wisam Dakka, Rishabh Dayal, and Panagiotis G. Ipeirotis. 2006. Automatic discovery of useful facet terms. In Proceedings of the SIGIR Workshop on Faceted Search."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835449.1835626"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484095"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2348283.2348296"},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the NAACL Conference on Human Language Technologies (HLT'10)","author":"Bona Fabio De","year":"2010","unstructured":"Fabio De Bona , Stefan Riezler , Keith Hall , Massimiliano Ciaramita , Ama\u00e7 Herda\u01e7delen , and Maria Holmqvist . 2010 . Learning dense models of query similarity from user click logs . In Proceedings of the NAACL Conference on Human Language Technologies (HLT'10) . Association for Computational Linguistics, Stroudsburg, PA, 474--482. http:\/\/dl.acm.org\/citation.cfm&quest;id= 1857999.1858070 Fabio De Bona, Stefan Riezler, Keith Hall, Massimiliano Ciaramita, Ama\u00e7 Herda\u01e7delen, and Maria Holmqvist. 2010. Learning dense models of query similarity from user click logs. In Proceedings of the NAACL Conference on Human Language Technologies (HLT'10). Association for Computational Linguistics, Stroudsburg, PA, 474--482. http:\/\/dl.acm.org\/citation.cfm&quest;id=1857999.1858070"},{"key":"e_1_2_1_19_1","series-title":"Lecture Notes in Computer Science","volume-title":"Proceedings of the 17th International Symposium on String Processing and Information Retrieval (SPIRE)","author":"Dupret Georges","unstructured":"Georges Dupret , Ricardo Zilleruelo-Ramos , and Sumio Fujita . 2010. Using related queries to improve Web search results ranking . In Proceedings of the 17th International Symposium on String Processing and Information Retrieval (SPIRE) . Lecture Notes in Computer Science , Vol. 6393 , Springer , Berlin , 213--224. Georges Dupret, Ricardo Zilleruelo-Ramos, and Sumio Fujita. 2010. Using related queries to improve Web search results ranking. In Proceedings of the 17th International Symposium on String Processing and Information Retrieval (SPIRE). Lecture Notes in Computer Science, Vol. 6393, Springer, Berlin, 213--224."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484069"},{"key":"e_1_2_1_21_1","volume-title":"Mathematical Programming Studies","volume":"8","author":"Fisher M. L.","unstructured":"M. L. Fisher , G. L. Nemhauser , and L. A. Wolsey . 1978. An analysis of approximations for maximizing submodular set functionsII. In Polyhedral Combinatorics, M. L. Balinski and A. J. Hoffman (Eds.) , Mathematical Programming Studies , Vol. 8 , Springer, Berlin Heidelberg, 73--87. DOI: http:\/\/dx.doi.org\/10.1007\/BFb0121195 10.1007\/BFb0121195 M. L. Fisher, G. L. Nemhauser, and L. A. Wolsey. 1978. An analysis of approximations for maximizing submodular set functionsII. In Polyhedral Combinatorics, M. L. Balinski and A. J. Hoffman (Eds.), Mathematical Programming Studies, Vol. 8, Springer, Berlin Heidelberg, 73--87. DOI: http:\/\/dx.doi.org\/10.1007\/BFb0121195"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1059981.1059982"},{"key":"e_1_2_1_23_1","volume-title":"Baeza-Yates","author":"Fujita Sumio","year":"2012","unstructured":"Sumio Fujita , Georges Dupret , and Ricardo A . Baeza-Yates . 2012 . Learning to rank query recommendations by semantic similarities. CoRR abs\/1204.2712 (2012). Sumio Fujita, Georges Dupret, and Ricardo A. Baeza-Yates. 2012. Learning to rank query recommendations by semantic similarities. CoRR abs\/1204.2712 (2012)."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1571941.1572003"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063576.2063981"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484055"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063576.2063619"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063576.2063599"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2396761.2398537"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484178"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the 20th Text Retrieval Conference (TREC'11)","author":"He Jiyin","year":"2011","unstructured":"Jiyin He , Vera Hollink , Corrado Boscarino , Arjen P. de Vries , and Roberto Cornacchia . 2011 . CWI at TREC 2011: Session, Web, and Medical . In Proceedings of the 20th Text Retrieval Conference (TREC'11) . Jiyin He, Vera Hollink, Corrado Boscarino, Arjen P. de Vries, and Roberto Cornacchia. 2011. CWI at TREC 2011: Session, Web, and Medical. In Proceedings of the 20th Text Retrieval Conference (TREC'11)."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.5555\/1793274.1793280"},{"key":"e_1_2_1_33_1","volume-title":"Advances in Kernel Methods","author":"Joachims Thorsten","unstructured":"Thorsten Joachims . 1999. Making large-scale support vector machine learning practical . In Advances in Kernel Methods , Bernhard Sch\u00f6lkopf , Christopher J. C. Burges, and Alexander J. Smola (Eds.), MIT Press , 169--184. http:\/\/dl.acm.org\/citation.cfm&quest;id=299094.299104. Thorsten Joachims. 1999. Making large-scale support vector machine learning practical. In Advances in Kernel Methods, Bernhard Sch\u00f6lkopf, Christopher J. C. Burges, and Alexander J. Smola (Eds.), MIT Press, 169--184. http:\/\/dl.acm.org\/citation.cfm&quest;id=299094.299104."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1229179.1229181"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458176"},{"key":"e_1_2_1_36_1","volume-title":"Overview of the TREC 2010 session track. In Proceedings of the 19th Text Retrieval Conference (TREC).","author":"Kanoulas Evangelos","year":"2010","unstructured":"Evangelos Kanoulas , Ben Carterette , Paul D. Clough , and Mark Sanderson . 2010 . Overview of the TREC 2010 session track. In Proceedings of the 19th Text Retrieval Conference (TREC). Evangelos Kanoulas, Ben Carterette, Paul D. Clough, and Mark Sanderson. 2010. Overview of the TREC 2010 session track. In Proceedings of the 19th Text Retrieval Conference (TREC)."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2010056"},{"key":"e_1_2_1_38_1","volume-title":"Overview of the TREC 2011 session track. In Proceedings of the 20th Text Retrieval Conference (TREC).","author":"Kanoulas Evangelos","year":"2011","unstructured":"Evangelos Kanoulas , Mark M. Hall , Paul Clough , Ben Carterette , and Mark Sanderson . 2011 b. Overview of the TREC 2011 session track. In Proceedings of the 20th Text Retrieval Conference (TREC). Evangelos Kanoulas, Mark M. Hall, Paul Clough, Ben Carterette, and Mark Sanderson. 2011b. Overview of the TREC 2011 session track. In Proceedings of the 20th Text Retrieval Conference (TREC)."},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the International ACM SIGIR Conference on Research and Development Information Retrieval (SIGIR'06)","author":"Kohlschutter Christian","year":"2006","unstructured":"Christian Kohlschutter , Paul-Alexandru Chirita , and Wolfgang Nejdl . 2006 . Using link analysis to identify aspects in faceted web search . In Proceedings of the International ACM SIGIR Conference on Research and Development Information Retrieval (SIGIR'06) . Christian Kohlschutter, Paul-Alexandru Chirita, and Wolfgang Nejdl. 2006. Using link analysis to identify aspects in faceted web search. In Proceedings of the International ACM SIGIR Conference on Research and Development Information Retrieval (SIGIR'06)."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484097"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2009922"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2187836.2187903"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2348283.2348456"},{"key":"e_1_2_1_44_1","volume-title":"Proceedings of the 20th Text Retrieval Conference (TREC).","author":"Liu Chang","unstructured":"Chang Liu , Si Sun , Michael J. Cole , and Nicholas J. Belkin . 2011. Rutgers at the TREC 2011 session track . In Proceedings of the 20th Text Retrieval Conference (TREC). Chang Liu, Si Sun, Michael J. Cole, and Nicholas J. Belkin. 2011. Rutgers at the TREC 2011 session track. In Proceedings of the 20th Text Retrieval Conference (TREC)."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835449.1835457"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/1816123.1816134"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935875"},{"key":"e_1_2_1_48_1","volume-title":"Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI'10)","author":"Ma Hao","year":"2010","unstructured":"Hao Ma , Michael R. Lyu , and Irwin King . 2010 . Diversifying query suggestion results . In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI'10) . AAAI Press. Hao Ma, Michael R. Lyu, and Irwin King. 2010. Diversifying query suggestion results. In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI'10). AAAI Press."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/1121949.1121979"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357242"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989342"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484084"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1670564.1670572"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148320"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/1081870.1081899"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484089"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.5555\/3120137.3120149"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063576.2063623"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/2339530.2339642"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772776"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484031"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772780"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/1871437.1871586"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835449.1835524"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.5555\/3104322.3104447"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/2348283.2348300"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2010025"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/WI-IAT.2009.33"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/1871437.1871565"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/1242572.1242576"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/1121949.1121978"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2007.09.011"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208312"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390310"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/860435.860440"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835449.1835511"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/1135777.1135858"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2629553","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2629553","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T07:01:18Z","timestamp":1750230078000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2629553"}},"subtitle":["Improving Whole-Session Relevance"],"short-title":[],"issued":{"date-parts":[[2014,10,28]]},"references-count":77,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2014,10,28]]}},"alternative-id":["10.1145\/2629553"],"URL":"https:\/\/doi.org\/10.1145\/2629553","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,10,28]]},"assertion":[{"value":"2013-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-10-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}