{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T05:14:43Z","timestamp":1784178883956,"version":"3.55.0"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"1","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2008,12]]},"abstract":"<jats:p>\n            A range of methods for measuring the effectiveness of information retrieval systems has been proposed. These are typically intended to provide a quantitative single-value summary of a document ranking relative to a query. However, many of these measures have failings. For example, recall is not well founded as a measure of satisfaction, since the user of an actual system cannot judge recall. Average precision is derived from recall, and suffers from the same problem. In addition, average precision lacks key stability properties that are needed for robust experiments. In this article, we introduce a new effectiveness metric,\n            <jats:italic>rank-biased precision<\/jats:italic>\n            , that avoids these problems. Rank-biased pre-cision is derived from a simple model of user behavior, is robust if answer rankings are extended to greater depths, and allows accurate quantification of experimental uncertainty, even when only partial relevance judgments are available.\n          <\/jats:p>","DOI":"10.1145\/1416950.1416952","type":"journal-article","created":{"date-parts":[[2008,12,23]],"date-time":"2008-12-23T14:15:53Z","timestamp":1230041753000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":457,"title":["Rank-biased precision for measurement of retrieval effectiveness"],"prefix":"10.1145","volume":"27","author":[{"given":"Alistair","family":"Moffat","sequence":"first","affiliation":[{"name":"The University of Melbourne, Victoria, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Justin","family":"Zobel","sequence":"additional","affiliation":[{"name":"RMIT University and NICTA Victoria Research Laboratory, Victoria, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2008,12,23]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076109"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148263"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076134"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.291019"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1008992.1009000"},{"key":"e_1_2_1_6_1","first-page":"53","article-title":"Retrieval system evaluation. In TREC: Experiment and Evaluation in Information Retrieval. MIT Press, Cambridge, MA","volume":"3","author":"Buckley C.","year":"2005","unstructured":"Buckley , C. and Voorhees , E. M. 2005 . Retrieval system evaluation. In TREC: Experiment and Evaluation in Information Retrieval. MIT Press, Cambridge, MA , Chapter 3 , 53 -- 75 . Buckley, C. and Voorhees, E. M. 2005. Retrieval system evaluation. In TREC: Experiment and Evaluation in Information Retrieval. MIT Press, Cambridge, MA, Chapter 3, 53--75.","journal-title":"Chapter"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277755"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148219"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.5090190108"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.4630240204"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148262"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.10408"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(91)90046-O"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(94)00047-7"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199601)47:1%3C37::AID-ASI4%3E3.3.CO;2-I"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076052"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076063"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1025603704680"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(92)90006-L"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2005.01.004"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(2000)51:9%3C834::AID-ASI60%3E3.3.CO;2-T"},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the Twenty-Eighth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM Press","author":"Marchionini G.","unstructured":"Marchionini , G. , Moffat , A. , Tate , J. , Baeza-Yates , R. , and Ziviani , N. , Eds . 2005 . Proceedings of the Twenty-Eighth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM Press , New York, NY. Marchionini, G., Moffat, A., Tate, J., Baeza-Yates, R., and Ziviani, N., Eds. 2005. Proceedings of the Twenty-Eighth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM Press, New York, NY."},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the International Conference on Internet Computing, H. R. Arabnia, O. Droegehorn, and S. Chatterjee, Eds. CSREA Press","author":"Meng X.","unstructured":"Meng , X. and Chen , Z . 2004. On user-oriented measurements of effectiveness of Web information retrieval systems . In Proceedings of the International Conference on Internet Computing, H. R. Arabnia, O. Droegehorn, and S. Chatterjee, Eds. CSREA Press , Las Vegas, NV, 527--533. Meng, X. and Chen, Z. 2004. On user-oriented measurements of effectiveness of Web information retrieval systems. In Proceedings of the International Conference on Internet Computing, H. R. Arabnia, O. Droegehorn, and S. Chatterjee, Eds. CSREA Press, Las Vegas, NV, 527--533."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199709)48:9%3C810::AID-ASI6%3E3.0.CO;2-U"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277806"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/65943.65945"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-31871-2_22"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277756"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076064"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/215206.215351"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.4630370511"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199404)45:3%3C207::AID-ASI10%3E3.0.CO;2-1"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(92)90005-K"},{"key":"e_1_2_1_35_1","volume-title":"Information Retrieval","author":"van Rijsbergen C. J.","unstructured":"van Rijsbergen , C. J. 1979. Information Retrieval , 2 nd ed. Butterworths , London, U.K. van Rijsbergen, C. J. 1979. Information Retrieval, 2nd ed. Butterworths, London, U.K.","edition":"2"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.5555\/648264.753539"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390334.1390346"},{"key":"e_1_2_1_38_1","volume-title":"Managing Gigabytes: Compressing and Indexing Documents and Images","author":"Witten I. H.","year":"1999","unstructured":"Witten , I. H. , Moffat , A. , and Bell , T. C . 1999 . Managing Gigabytes: Compressing and Indexing Documents and Images , 2 nd ed. Morgan Kaufmann , San Francisco, CA . Witten, I. H., Moffat, A., and Bell, T. C. 1999. Managing Gigabytes: Compressing and Indexing Documents and Images, 2nd ed. Morgan Kaufmann, San Francisco, CA.","edition":"2"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.291014"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1416950.1416952","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,29]],"date-time":"2022-12-29T07:36:24Z","timestamp":1672299384000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1416950.1416952"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,12]]},"references-count":39,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2008,12]]}},"alternative-id":["10.1145\/1416950.1416952"],"URL":"https:\/\/doi.org\/10.1145\/1416950.1416952","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2008,12]]},"assertion":[{"value":"2005-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2008-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2008-12-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}