{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,16]],"date-time":"2026-01-16T19:01:20Z","timestamp":1768590080583,"version":"3.49.0"},"publisher-location":"Berlin, Heidelberg","reference-count":18,"publisher":"Springer Berlin Heidelberg","isbn-type":[{"value":"9783642159978","type":"print"},{"value":"9783642159985","type":"electronic"}],"license":[{"start":{"date-parts":[[2010,1,1]],"date-time":"2010-01-01T00:00:00Z","timestamp":1262304000000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2010]]},"DOI":"10.1007\/978-3-642-15998-5_13","type":"book-chapter","created":{"date-parts":[[2010,9,9]],"date-time":"2010-09-09T23:59:35Z","timestamp":1284076775000},"page":"112-123","source":"Crossref","is-referenced-by-count":10,"title":["Tie-Breaking Bias: Effect of an Uncontrolled Parameter on Information Retrieval Evaluation"],"prefix":"10.1007","author":[{"given":"Guillaume","family":"Cabanac","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gilles","family":"Hubert","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohand","family":"Boughanem","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Claude","family":"Chrisment","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","reference":[{"issue":"4","key":"13_CR1","doi-asserted-by":"publisher","first-page":"439","DOI":"10.1177\/0165551507086989","volume":"34","author":"S. Robertson","year":"2008","unstructured":"Robertson, S.: On the history of evaluation in IR. J. Inf. Sci.\u00a034(4), 439\u2013456 (2008)","journal-title":"J. Inf. Sci."},{"key":"13_CR2","doi-asserted-by":"crossref","unstructured":"Harman, D.K. (ed.): TREC-1: Proceedings of the First Text REtrieval Conference, Gaithersburg, MD, USA, NIST (February 1993)","DOI":"10.6028\/NIST.SP.500-207"},{"key":"13_CR3","volume-title":"TREC: Experiment and Evaluation in Information Retrieval","author":"E.M. Voorhees","year":"2005","unstructured":"Voorhees, E.M., Harman, D.K.: TREC: Experiment and Evaluation in Information Retrieval. MIT Press, Cambridge (2005)"},{"key":"13_CR4","unstructured":"NIST: README file for trec_eval\u00a08.1, \n                    \n                      http:\/\/trec.nist.gov\/trec_eval"},{"key":"13_CR5","unstructured":"Kando, N., Kuriyama, K., Nozue, T., Eguchi, K., Kato, H., Hidaka, S.: Overview of IR Tasks at the First NTCIR Workshop. In: Proceedings of the First NTCIR Workshop on Research in Japanese Text Retrieval and Term Recognition, NACSIS, pp. 11\u201344 (1999)"},{"issue":"12","key":"13_CR6","doi-asserted-by":"publisher","first-page":"1067","DOI":"10.1002\/asi.1164","volume":"52","author":"C. Peters","year":"2001","unstructured":"Peters, C., Braschler, M.: European Research Letter \u2013 Cross-Language System Evaluation: the CLEF Campaigns. J. Am. Soc. Inf. Sci. Technol.\u00a052(12), 1067\u20131072 (2001)","journal-title":"J. Am. Soc. Inf. Sci. Technol."},{"key":"13_CR7","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"publisher","first-page":"581","DOI":"10.1007\/978-3-540-30222-3_56","volume-title":"Comparative Evaluation of Multilingual Information Access Systems","author":"P. Clough","year":"2004","unstructured":"Clough, P., Sanderson, M.: The CLEF 2003 Cross Language Image Retrieval Track. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol.\u00a03237, pp. 581\u2013593. Springer, Heidelberg (2004)"},{"key":"13_CR8","first-page":"315","volume-title":"SIGIR 1998: Proceedings of the 21st Annual International ACM SIGIR Conference","author":"E.M. Voorhees","year":"1998","unstructured":"Voorhees, E.M.: Variations in Relevance Judgments and the Measurement of Retrieval Effectiveness. In: SIGIR 1998: Proceedings of the 21st Annual International ACM SIGIR Conference, pp. 315\u2013323. ACM, New York (1998)"},{"key":"13_CR9","first-page":"307","volume-title":"SIGIR 1998: Proceedings of the 21st Annual International ACM SIGIR Conference","author":"J. Zobel","year":"1998","unstructured":"Zobel, J.: How Reliable are the Results of large-scale Information Retrieval Experiments? In: SIGIR 1998: Proceedings of the 21st Annual International ACM SIGIR Conference, pp. 307\u2013314. ACM Press, New York (1998)"},{"key":"13_CR10","unstructured":"Buckley, C., Voorhees, E.M.: Retrieval System Evaluation. In: [3], ch. \u00a03, pp. 53\u201375"},{"key":"13_CR11","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511809071","volume-title":"Introduction to Information Retrieval","author":"C.D. Manning","year":"2008","unstructured":"Manning, C.D., Raghavan, P., Sch\u00fctze, H.: Introduction to Information Retrieval. Cambridge University Press, Cambridge (July 2008)"},{"key":"13_CR12","doi-asserted-by":"crossref","unstructured":"Voorhees, E.M.: Overview of the TREC 2004 Robust Track. In: Voorhees, E.M., Buckland, L.P. (eds.) TREC 2004: Proceedings of the 13th Text REtrieval Conference, Gaithersburg, MD, USA, NIST (2004)","DOI":"10.6028\/NIST.SP.500-261"},{"key":"13_CR13","first-page":"162","volume-title":"SIGIR 2005: Proceedings of the 28th annual international ACM SIGIR conference","author":"M. Sanderson","year":"2005","unstructured":"Sanderson, M., Zobel, J.: Information Retrieval System Evaluation: Effort, Sensitivity, and Reliability. In: SIGIR 2005: Proceedings of the 28th annual international ACM SIGIR conference, pp. 162\u2013169. ACM, New York (2005)"},{"key":"13_CR14","first-page":"329","volume-title":"SIGIR 1993: Proceedings of the 16th annual international ACM SIGIR conference","author":"D. Hull","year":"1993","unstructured":"Hull, D.: Using Statistical Testing in the Evaluation of Retrieval Experiments. In: SIGIR 1993: Proceedings of the 16th annual international ACM SIGIR conference, pp. 329\u2013338. ACM Press, New York (1993)"},{"issue":"3","key":"13_CR15","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1145\/65943.65945","volume":"7","author":"V. Raghavan","year":"1989","unstructured":"Raghavan, V., Bollmann, P., Jung, G.S.: A critical investigation of recall and precision as measures of retrieval system performance. ACM Trans. Inf. Syst.\u00a07(3), 205\u2013229 (1989)","journal-title":"ACM Trans. Inf. Syst."},{"key":"13_CR16","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"publisher","first-page":"414","DOI":"10.1007\/978-3-540-78646-7_38","volume-title":"Advances in Information Retrieval","author":"F. McSherry","year":"2008","unstructured":"McSherry, F., Najork, M.: Computing Information Retrieval Performance Measures Efficiently in the Presence of Tied Scores. In: Macdonald, C., Ounis, I., Plachouras, V., Ruthven, I., White, R.W. (eds.) ECIR 2008. LNCS, vol.\u00a04956, pp. 414\u2013421. Springer, Heidelberg (2008)"},{"key":"13_CR17","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"publisher","first-page":"483","DOI":"10.1007\/11551362_46","volume-title":"Research and Advanced Technology for Digital Libraries","author":"G.M. Nunzio Di","year":"2005","unstructured":"Di Nunzio, G.M., Ferro, N.: DIRECT: A System for Evaluating Information Access Components of Digital Libraries. In: Rauber, A., Christodoulakis, S., Tjoa, A.M. (eds.) ECDL 2005. LNCS, vol.\u00a03652, pp. 483\u2013484. Springer, Heidelberg (2005)"},{"issue":"2","key":"13_CR18","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1145\/1328964.1328974","volume":"41","author":"T. Joachims","year":"2007","unstructured":"Joachims, T., Li, H., Liu, T.Y., Zhai, C.: Learning to Rank for Information Retrieval (LR4IR 2007). SIGIR Forum\u00a041(2), 58\u201362 (2007)","journal-title":"SIGIR Forum"}],"container-title":["Lecture Notes in Computer Science","Multilingual and Multimodal Information Access Evaluation"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-642-15998-5_13","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,5,19]],"date-time":"2019-05-19T18:02:43Z","timestamp":1558288963000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/978-3-642-15998-5_13"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2010]]},"ISBN":["9783642159978","9783642159985"],"references-count":18,"URL":"https:\/\/doi.org\/10.1007\/978-3-642-15998-5_13","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"value":"0302-9743","type":"print"},{"value":"1611-3349","type":"electronic"}],"subject":[],"published":{"date-parts":[[2010]]}}}