{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,6,9]],"date-time":"2024-06-09T10:49:37Z","timestamp":1717930177661},"reference-count":26,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2015,12,28]],"date-time":"2015-12-28T00:00:00Z","timestamp":1451260800000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Inf Retrieval J"],"published-print":{"date-parts":[[2016,6]]},"DOI":"10.1007\/s10791-015-9276-9","type":"journal-article","created":{"date-parts":[[2015,12,27]],"date-time":"2015-12-27T23:01:25Z","timestamp":1451257285000},"page":"230-255","update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["The strange case of reproducibility versus representativeness in contextual suggestion test collections"],"prefix":"10.1007","volume":"19","author":[{"given":"Thaer","family":"Samar","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alejandro","family":"Bellog\u00edn","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Arjen P.","family":"de Vries","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2015,12,28]]},"reference":[{"key":"9276_CR1","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1007\/978-0-387-85820-3_7","volume-title":"Recommender systems handbook","author":"G Adomavicius","year":"2011","unstructured":"Adomavicius, G., & Tuzhilin, A. (2011). Context-aware recommender systems. In F. Ricci, L. Rokach, B. Shapira, F. Ricci, L. Rokach, B. Shapira, P. B. Kantor, & P. B. Kantor (Eds.), Recommender systems handbook (pp. 217\u2013253). Boston, MA: Springer. doi: 10.1007\/978-0-387-85820-3_7 . (chapter 6)."},{"issue":"1","key":"9276_CR2","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1145\/2215676.2215678","volume":"46","author":"J Allan","year":"2012","unstructured":"Allan, J., Croft, W. B., Moffat, A., & Sanderson, M. (2012). Frontiers, challenges, and opportunities for information retrieval: Report from SWIRL 2012 the second strategic workshop on information retrieval in lorne. SIGIR Forum, 46(1), 2\u201332. doi: 10.1145\/2215676.2215678 .","journal-title":"SIGIR Forum"},{"key":"9276_CR3","doi-asserted-by":"publisher","unstructured":"Bellog\u00edn, A., Samar, T., de\u00a0Vries, A. P., & Said, A. (2014). Challenges on combining open web and dataset evaluation results: The case of the contextual suggestion track. In M. de\u00a0Rijke, T. Kenter, A. P. de\u00a0Vries, C. Zhai, F. de\u00a0Jong, K. Radinsky, & K. Hofmann (Eds.), Advances in information retrieval-36th European conference on IR research, proceedings, Springer, Lecture Notes in Computer Science (Vol. 8416, pp. 430\u2013436). Amsterdam, The Netherlands: ECIR 2014. doi: 10.1007\/978-3-319-06028-6_37 .","DOI":"10.1007\/978-3-319-06028-6_37"},{"key":"9276_CR4","doi-asserted-by":"crossref","unstructured":"Davison, B. D. (2000). Topical locality in the web. In Proceedings of the 23rd annual international conference on research and development in information retrieval (SIGIR 2000) (pp. 272\u2013279). ACM Press.","DOI":"10.1145\/345508.345597"},{"key":"9276_CR5","doi-asserted-by":"publisher","unstructured":"Dean-Hall, A., & Clarke, C. L. A. (2015). The power of contextual suggestion. In A. Hanbury, G. Kazai, A. Rauber, N. Fuhr (Eds.), Advances in information retrieval-37th European conference on IR research, ECIR 2015, proceedings, Lecture Notes in Computer Science (Vol. 9022, pp. 352\u2013357), Vienna, Austria, March 29\u2013April 2, 2015. doi: 10.1007\/978-3-319-16354-3_39 .","DOI":"10.1007\/978-3-319-16354-3_39"},{"key":"9276_CR6","unstructured":"Dean-Hall, A., Clarke, C. L. A., Kamps, J., Thomas, P., & Voorhees, E. M. (2012). Overview of the TREC 2012 contextual suggestion track. In E. M. Voorhees, L. P. Buckland (Eds.), Proceedings of the twenty-first text retrieval conference, TREC 2012, National Institute of Standards and Technology (NIST), Vol Special Publication 500-298, Gaithersburg, Maryland, USA, November 6\u20139, 2012. http:\/\/trec.nist.gov\/pubs\/trec21\/papers\/CONTEXTUAL12.overview.pdf ."},{"key":"9276_CR7","unstructured":"Dean-Hall, A., Clarke, C. L. A., Kamps, J., & Thomas, P. (2013). Evaluating contextual suggestion. In Proceedings of the fifth international workshop on evaluating information access (EVIA 2013)."},{"key":"9276_CR8","unstructured":"Dean-Hall, A., Clarke, C. L. A., Simone, N., Kamps, J., Thomas, P., & Voorhees, E. M. (2013). Overview of the TREC 2013 contextual suggestion track. In E. M. Voorhees (Ed.), Proceedings of the twenty-second text retrieval conference, TREC 2013, National Institute of Standards and Technology (NIST), Vol Special Publication 500-302, Gaithersburg, Maryland, USA, November 19\u201322, 2013. http:\/\/trec.nist.gov\/pubs\/trec22\/papers\/CONTEXT.OVERVIEW.pdf ."},{"key":"9276_CR9","unstructured":"Dean-Hall, A., Clarke, C. L. A., Kamps, J., Thomas, P., Voorhees, E. M. (2014). Overview of the TREC 2014 contextual suggestion track. In E. M. Voorhees, A. Ellis, (Eds.), Proceedings of the twenty-third text retrieval conference, TREC 2014, National Institute of Standards and Technology (NIST), vol Special Publication 500-308, Gaithersburg, Maryland, USA, November 19\u201321, 2014. http:\/\/trec.nist.gov\/pubs\/trec23\/papers\/overview-context.pdf"},{"key":"9276_CR10","doi-asserted-by":"publisher","unstructured":"Deveaud, R., Albakour, M. D., Macdonald, C., & Ounis, I. (2014). On the importance of venue-dependent features for learning to rank contextual suggestions. In Proceedings of the 23rd ACM international conference on conference on information and knowledge management, CIKM \u201914 (pp. 1827\u20131830). New York, NY, USA: ACM. doi: 10.1145\/2661829.2661956 .","DOI":"10.1145\/2661829.2661956"},{"issue":"11\u201316","key":"9276_CR11","doi-asserted-by":"publisher","first-page":"1321","DOI":"10.1016\/S1389-1286(99)00024-9","volume":"31","author":"D Hawking","year":"1999","unstructured":"Hawking, D., Craswell, N., Thistlewaite, P. B., & Harman, D. (1999). Results and challenges in web search evaluation. Computer Networks, 31(11\u201316), 1321\u20131330. doi: 10.1016\/S1389-1286(99)00024-9 .","journal-title":"Computer Networks"},{"issue":"1","key":"9276_CR12","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1023\/A:1011468107287","volume":"4","author":"D Hawking","year":"2001","unstructured":"Hawking, D., Craswell, N., Bailey, P., & Griffiths, K. (2001). Measuring search engine quality. Information Retrieval, 4(1), 33\u201359. doi: 10.1023\/A:1011468107287 .","journal-title":"Information Retrieval"},{"key":"9276_CR13","unstructured":"Hubert, G., Cabanac, G., Pinel-Sauvagnat, K., Palacio, D., & Sallaberry, C. (2013). IRIT, GeoComp, and LIUPPA at the TREC 2013 contextual suggestion track. In Proceedings of TREC."},{"issue":"1\u20132","key":"9276_CR14","first-page":"1","volume":"3","author":"D Kelly","year":"2009","unstructured":"Kelly, D. (2009). Methods for evaluating interactive information retrieval systems with users. Foundations and Trends in Information Retrieval, 3(1\u20132), 1\u2013224.","journal-title":"Foundations and Trends in Information Retrieval"},{"key":"9276_CR15","doi-asserted-by":"publisher","unstructured":"Kraaij, W., Westerveld, T., & Hiemstra, D. (2002). The importance of prior probabilities for entry page search. In SIGIR 2002: Proceedings of the 25th annual international ACM SIGIR conference on research and development in information retrieval (pp. 27\u201334). Tampere, Finland: ACM, August 11\u201315, 2002. doi: 10.1145\/564376.564383 .","DOI":"10.1145\/564376.564383"},{"key":"9276_CR16","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1007\/978-0-387-85820-3_3","volume-title":"Recommender systems handbook","author":"P Lops","year":"2011","unstructured":"Lops, P., de Gemmis, M., & Semeraro, G. (2011). Content-based recommender systems: State of the art and trends. In F. Ricci, L. Rokach, B. Shapira, F. Ricci, L. Rokach, B. Shapira, P. B. Kantor, & P. B. Kantor (Eds.), Recommender systems handbook (pp. 73\u2013105). Boston, MA: Springer. doi: 10.1007\/978-0-387-85820-3_3 . (chapter 3)."},{"issue":"4\u20135","key":"9276_CR17","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1561\/1500000023","volume":"6","author":"M Melucci","year":"2012","unstructured":"Melucci, M. (2012). Contextual search: A computational framework. Foundations and Trends in Information Retrieval, 6(4\u20135), 257\u2013405.","journal-title":"Foundations and Trends in Information Retrieval"},{"key":"9276_CR18","volume-title":"C4.5: Programs for machine learning","author":"R Quinlan","year":"1993","unstructured":"Quinlan, R. (1993). C4.5: Programs for machine learning. San Mateo, CA: Morgan Kaufmann Publishers."},{"key":"9276_CR19","doi-asserted-by":"crossref","unstructured":"Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., Riedl, J. (1994). Grouplens: An open architecture for collaborative filtering of netnews. In CSCW (pp. 175\u2013186).","DOI":"10.1145\/192844.192905"},{"key":"9276_CR20","doi-asserted-by":"publisher","unstructured":"Sanderson, M., Turpin, A., Zhang, Y., & Scholer, F. (2012). Differences in effectiveness across sub-collections. In: X. Chen, G. Lebanon, H. Wang, & M. J. Zaki (Eds.), 21st ACM international conference on information and knowledge management, CIKM\u201912 (pp. 1965\u20131969) Maui, HI, USA: ACM, October 29\u2013November 02, 2012. doi: 10.1145\/2396761.2398553 .","DOI":"10.1145\/2396761.2398553"},{"key":"9276_CR21","doi-asserted-by":"publisher","unstructured":"Santos, R. L. T., Macdonald, C., & Ounis, I. (2011). Effectiveness beyond the first crawl tier. In C. Macdonald, I. Ounis I. Ruthven (Eds.), Proceedings of the 20th ACM conference on information and knowledge management, CIKM 2011 (pp. 1937\u20131940). Glasgow, United Kingdom: ACM, October 24\u201328, 2011. doi: 10.1145\/2063576.2063859 .","DOI":"10.1145\/2063576.2063859"},{"key":"9276_CR22","doi-asserted-by":"publisher","unstructured":"Sappelli, M., Verberne, S., & Kraaij, W. (2013). Recommending personalized touristic sights using google places. In: Proceedings of the 36th international ACM SIGIR conference on research and development in information retrieval, SIGIR \u201913 (pp. 781\u2013784). New York, NY, USA: ACM. doi: 10.1145\/2484028.2484155 .","DOI":"10.1145\/2484028.2484155"},{"key":"9276_CR23","doi-asserted-by":"publisher","unstructured":"Scholer, F., Turpin, A., & Sanderson, M. (2011). Quantifying test collection quality based on the consistency of relevance judgements. In: W. Ma, J. Nie, R. A. Baeza-Yates, T. Chua, W. B. Croft (Eds.), Proceeding of the 34th international ACM SIGIR conference on research and development in information retrieval, SIGIR 2011 (pp. 1063\u20131072) Beijing, China: ACM, July 25\u201329, 2011. doi: 10.1145\/2009916.2010057 .","DOI":"10.1145\/2009916.2010057"},{"key":"9276_CR24","doi-asserted-by":"publisher","unstructured":"Voorhees, E. M., & Buckley, C. (2002). The effect of topic set size on retrieval experiment error. In Proceedings of the 25th annual international ACM SIGIR conference on research and development in information retrieval, SIGIR \u201902 (pp. 316\u2013323). New York, NY, USA: ACM. doi: 10.1145\/564376.564432 .","DOI":"10.1145\/564376.564432"},{"key":"9276_CR25","unstructured":"Yang, P., & Fang, H. (2012). An Exploration of ranking-based strategy for contextual suggestions. In Proceedings of TREC."},{"key":"9276_CR26","doi-asserted-by":"publisher","unstructured":"Yang, P., & Fang, H. (2013). Opinion-based user profile modeling for contextual suggestions. In Proceedings of the 2013 conference on the theory of information retrieval ICTIR \u201913 (pp. 18:80\u201318:83). New York, NY, USA: ACM. doi: 10.1145\/2499178.2499191 .","DOI":"10.1145\/2499178.2499191"}],"container-title":["Information Retrieval Journal"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10791-015-9276-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s10791-015-9276-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10791-015-9276-9","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,3,26]],"date-time":"2019-03-26T15:07:25Z","timestamp":1553612845000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s10791-015-9276-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,12,28]]},"references-count":26,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2016,6]]}},"alternative-id":["9276"],"URL":"https:\/\/doi.org\/10.1007\/s10791-015-9276-9","relation":{},"ISSN":["1386-4564","1573-7659"],"issn-type":[{"value":"1386-4564","type":"print"},{"value":"1573-7659","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,12,28]]}}}