{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T20:49:11Z","timestamp":1771706951813,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":32,"publisher":"ACM","license":[{"start":{"date-parts":[[2010,7,19]],"date-time":"2010-07-19T00:00:00Z","timestamp":1279497600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2010,7,19]]},"DOI":"10.1145\/1835449.1835551","type":"proceedings-article","created":{"date-parts":[[2010,7,20]],"date-time":"2010-07-20T15:39:10Z","timestamp":1279640350000},"page":"611-618","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":70,"title":["PRES"],"prefix":"10.1145","author":[{"given":"Walid","family":"Magdy","sequence":"first","affiliation":[{"name":"Dublin City University, Dublin, Ireland"}]},{"given":"Gareth J.F.","family":"Jones","sequence":"additional","affiliation":[{"name":"Dublin City University, Dublin, Ireland"}]}],"member":"320","published-online":{"date-parts":[[2010,7,19]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458235"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1183614.1183633"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277820"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458157"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.5555\/553876"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-03573-9_63"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/345508.345543"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148284"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1008992.1009000"},{"key":"e_1_3_2_1_10_1","first-page":"16","volume-title":"Proceedings of ECIR '08","author":"Carterette B.","year":"2008","unstructured":"Carterette , B. , Bennett , P. N. Chickering , D. M. , and Dumais , S. T . Here or There: Preference Judgments for Relevance . In Proceedings of ECIR '08 , pages 16 -- 27 , 2008 . Carterette, B., Bennett, P. N. Chickering, D. M., and Dumais, S. T. Here or There: Preference Judgments for Relevance. In Proceedings of ECIR '08, pages 16--27, 2008."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/275537.275544"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/160688.160758"},{"key":"e_1_3_2_1_13_1","volume-title":"Proceedings of the 4th NTCIR Workshop","author":"Fujii A.","year":"2004","unstructured":"Fujii , A. , Iwayama , M. , and Kando , N . Overview of Patent Retrieval Task at NTCIR-4 . In Proceedings of the 4th NTCIR Workshop , 2004 . Fujii, A., Iwayama, M., and Kando, N. Overview of Patent Retrieval Task at NTCIR-4. In Proceedings of the 4th NTCIR Workshop, 2004."},{"key":"e_1_3_2_1_14_1","first-page":"60","volume-title":"Proceedings of the 2nd EVIA Workshop","author":"Graf E.","year":"2008","unstructured":"Graf , E. , and Azzopardi , L . A methodology for building a patent test collection for prior art search . In Proceedings of the 2nd EVIA Workshop , pages 60 -- 71 , 2008 . Graf, E., and Azzopardi, L. A methodology for building a patent test collection for prior art search. In Proceedings of the 2nd EVIA Workshop, pages 60--71, 2008."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141753.1141818"},{"key":"e_1_3_2_1_16_1","first-page":"24","volume-title":"INEX 2007 evaluation measures. In Proceedings of INEX '07","author":"Kamps J.","year":"2007","unstructured":"Kamps , J. , Pehcevski , J. , Kazai , G. , Lalmas , M. , and Robertson , S . INEX 2007 evaluation measures. In Proceedings of INEX '07 , pages 24 -- 33 , 2007 . Kamps, J., Pehcevski, J., Kazai, G., Lalmas, M., and Robertson, S. INEX 2007 evaluation measures. In Proceedings of INEX '07, pages 24--33, 2007."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.2307\/2332226"},{"key":"e_1_3_2_1_18_1","first-page":"27","article-title":"Recent developments in the evaluation of information retrieval systems: moving toward diversity and practical applications","volume":"32","author":"Mandl T","year":"2008","unstructured":"Mandl , T . Recent developments in the evaluation of information retrieval systems: moving toward diversity and practical applications . Informatica , 32 : 27 -- 38 , 2008 . Mandl, T. Recent developments in the evaluation of information retrieval systems: moving toward diversity and practical applications. Informatica, 32:27--38, 2008.","journal-title":"Informatica"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1416950.1416952"},{"key":"e_1_3_2_1_20_1","volume-title":"Overview of the TREC 2008 legal track. In Proceedings of TREC 2008","author":"Oard D. W.","year":"2008","unstructured":"Oard , D. W. , Hedin , B. , Tomlinson , S. , and Baron , J. R . Overview of the TREC 2008 legal track. In Proceedings of TREC 2008 , 2008 . Oard, D. W., Hedin, B., Tomlinson, S., and Baron, J. R. Overview of the TREC 2008 legal track. In Proceedings of TREC 2008, 2008."},{"key":"e_1_3_2_1_21_1","volume-title":"Information Retrieval","author":"van Rijsbergen C. J.","year":"1979","unstructured":"van Rijsbergen , C. J. Information Retrieval , 2 nd edition. Butterworths , 1979 . van Rijsbergen, C. J. Information Retrieval, 2nd edition. Butterworths, 1979.","edition":"2"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1108\/eb026466"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390334.1390453"},{"key":"e_1_3_2_1_24_1","volume-title":"Information storage and retrieval","author":"Rocchio J.","year":"1964","unstructured":"Rocchio J. Performance indices for document retrieval systems . In Information storage and retrieval , Computation Laboratory of Harvard University , Cambridge, MA , 1964 . Rocchio J. Performance indices for document retrieval systems. In Information storage and retrieval, Computation Laboratory of Harvard University, Cambridge, MA, 1964."},{"key":"e_1_3_2_1_25_1","volume-title":"Proceedings of CLEF '09","author":"Roda G.","year":"2009","unstructured":"Roda G. , Tait J. , Piroi F. , and Zenz V . CLEF-IP 2009: retrieval experiments in the Intellectual Property domain . In Proceedings of CLEF '09 , 2009 . Roda G., Tait J., Piroi F., and Zenz V. CLEF-IP 2009: retrieval experiments in the Intellectual Property domain. In Proceedings of CLEF '09, 2009."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/511754.511764"},{"key":"e_1_3_2_1_27_1","volume-title":"Proceedings of TREC 2007","author":"Tomlinson S.","year":"2007","unstructured":"Tomlinson S. , Oard , D. W. , Baron , J. R. , and Thompson , P . Overview of the TREC 2007 Legal Track . In Proceedings of TREC 2007 , 2007 . Tomlinson S., Oard, D. W., Baron, J. R., and Thompson, P. Overview of the TREC 2007 Legal Track. In Proceedings of TREC 2007, 2007."},{"key":"e_1_3_2_1_28_1","first-page":"77","volume-title":"Proceedings of TREC 1999","author":"Voorhees E. M.","year":"1999","unstructured":"Voorhees , E. M. , and Tice , D. M . The TREC-8 Question Answering Track Evaluation . In Proceedings of TREC 1999 , pages 77 -- 82 , 1999 . Voorhees, E. M., and Tice, D. M. The TREC-8 Question Answering Track Evaluation. In Proceedings of TREC 1999, pages 77--82, 1999."},{"key":"e_1_3_2_1_29_1","first-page":"355","volume-title":"Proceedings of CLEF '02","author":"Voorhees E. M.","year":"2002","unstructured":"Voorhees , E. M. The Philosophy of Information Retrieval Evaluation. In Evaluation of Cross-Language Information Retrieval System , Proceedings of CLEF '02 , pages 355 -- 370 , 2002 . Voorhees, E. M. The Philosophy of Information Retrieval Evaluation. In Evaluation of Cross-Language Information Retrieval System, Proceedings of CLEF '02, pages 355--370, 2002."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1067268.1067272"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1645953.1646295"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458572.1458577"}],"event":{"name":"SIGIR '10: The 33rd International ACM SIGIR conference on research and development in Information Retrieval","location":"Geneva Switzerland","acronym":"SIGIR '10","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1835449.1835551","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1835449.1835551","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T11:39:31Z","timestamp":1750246771000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1835449.1835551"}},"subtitle":["a score metric for evaluating recall-oriented information retrieval applications"],"short-title":[],"issued":{"date-parts":[[2010,7,19]]},"references-count":32,"alternative-id":["10.1145\/1835449.1835551","10.1145\/1835449"],"URL":"https:\/\/doi.org\/10.1145\/1835449.1835551","relation":{},"subject":[],"published":{"date-parts":[[2010,7,19]]},"assertion":[{"value":"2010-07-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}