{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T23:05:32Z","timestamp":1787785532678,"version":"build-2784847793"},"reference-count":23,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2010,11,1]],"date-time":"2010-11-01T00:00:00Z","timestamp":1288569600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2010,11]]},"abstract":"<jats:p>Ranked lists are encountered in research and daily life and it is often of interest to compare these lists even when they are incomplete or have only some members in common. An example is document rankings returned for the same query by different search engines. A measure of the similarity between incomplete rankings should handle nonconjointness, weight high ranks more heavily than low, and be monotonic with increasing depth of evaluation; but no measure satisfying all these criteria currently exists. In this article, we propose a new measure having these qualities, namely rank-biased overlap (RBO). The RBO measure is based on a simple probabilistic user model. It provides monotonicity by calculating, at a given depth of evaluation, a base score that is non-decreasing with additional evaluation, and a maximum score that is nonincreasing. An extrapolated score can be calculated between these bounds if a point estimate is required. RBO has a parameter which determines the strength of the weighting to top ranks. We extend RBO to handle tied ranks and rankings of different lengths. Finally, we give examples of the use of the measure in comparing the results produced by public search engines and in assessing retrieval systems in the laboratory.<\/jats:p>","DOI":"10.1145\/1852102.1852106","type":"journal-article","created":{"date-parts":[[2010,11,23]],"date-time":"2010-11-23T10:00:38Z","timestamp":1290506438000},"page":"1-38","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":647,"title":["A similarity measure for indefinite rankings"],"prefix":"10.1145","volume":"28","author":[{"given":"William","family":"Webber","sequence":"first","affiliation":[{"name":"The University of Melbourne, Australia, Victoria, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alistair","family":"Moffat","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Australia, Victoria, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Justin","family":"Zobel","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Australia, Victoria, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2010,11,23]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2005.03.008"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.comnet.2005.10.020"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1111\/1467-842X.00110"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1008992.1009093"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1571941.1572017"},{"key":"e_1_2_1_6_1","volume-title":"Ordinal Methods for Behavioural Data Analysis","author":"Cliff N.","unstructured":"Cliff , N. 1996. Ordinal Methods for Behavioural Data Analysis . Lawrence Erlbaum Associates . Cliff, N. 1996. Ordinal Methods for Behavioural Data Analysis. Lawrence Erlbaum Associates."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1137\/S0895480102412856"},{"key":"e_1_2_1_8_1","unstructured":"Gibbons J. D. and Chakraborti S. 2003. Nonparametric Statistical Inference 4th Ed. CRC.  Gibbons J. D. and Chakraborti S. 2003. Nonparametric Statistical Inference 4th Ed. CRC."},{"key":"e_1_2_1_9_1","article-title":"Measures of association for cross classifications","volume":"49","author":"Goodman L. A.","year":"1954","unstructured":"Goodman , L. A. and Kruskal , W. H. 1954 . Measures of association for cross classifications . J. Am. Statis. Assoc. 49 , 268, 732--764. Goodman, L. A. and Kruskal, W. H. 1954. Measures of association for cross classifications. J. Am. Statis. Assoc. 49, 268, 732--764.","journal-title":"J. Am. Statis. Assoc."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.2307\/1269344"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_2_1_12_1","volume-title":"Rank Correlation Methods","author":"Kendall M. G.","unstructured":"Kendall , M. G. 1948. Rank Correlation Methods 1 st Ed. Charles Griffin , London . Kendall, M. G. 1948. Rank Correlation Methods 1st Ed. Charles Griffin, London.","edition":"1"},{"key":"e_1_2_1_13_1","volume-title":"Fundamental Algorithms","author":"Knuth D. E.","unstructured":"Knuth , D. E. 1997. The Art of Computer Programming , Vol. I : Fundamental Algorithms . 3 rd Ed. Addison Wesley , Reading, MA . Knuth, D. E. 1997. The Art of Computer Programming, Vol. I: Fundamental Algorithms. 3rd Ed. Addison Wesley, Reading, MA.","edition":"3"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/11581062_37"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273221.1273223"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04769-5_7"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1416950.1416952"},{"key":"e_1_2_1_18_1","unstructured":"Quade D. and Salama I. A. 1992. A survey of weighted rank correlation. In Order Statistics and Nonparametrics: Theory and Applications P. K. Sen and I. A. Salama Eds. Elsevier 213--224.  Quade D. and Salama I. A. 1992. A survey of weighted rank correlation. In Order Statistics and Nonparametrics: Theory and Applications P. K. Sen and I. A. Salama Eds. Elsevier 213--224."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-7152(98)00006-6"},{"key":"e_1_2_1_20_1","unstructured":"Tarsitano A. 2002. Nonlinear rank correlation. Departmental working paper Universit\u00f2 degli studi della Calabria.  Tarsitano A. 2002. Nonlinear rank correlation. Departmental working paper Universit\u00f2 degli studi della Calabria."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/952532.952693"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390334.1390435"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/984321.984322"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1852102.1852106","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1852102.1852106","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T08:08:18Z","timestamp":1750234098000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1852102.1852106"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2010,11]]},"references-count":23,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2010,11]]}},"alternative-id":["10.1145\/1852102.1852106"],"URL":"https:\/\/doi.org\/10.1145\/1852102.1852106","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2010,11]]},"assertion":[{"value":"2009-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2010-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2010-11-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}