{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T06:11:46Z","timestamp":1775283106211,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":27,"publisher":"ACM","license":[{"start":{"date-parts":[[2004,7,25]],"date-time":"2004-07-25T00:00:00Z","timestamp":1090713600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2004,7,25]]},"DOI":"10.1145\/1008992.1009001","type":"proceedings-article","created":{"date-parts":[[2004,10,7]],"date-time":"2004-10-07T17:39:48Z","timestamp":1097170788000},"page":"33-40","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":51,"title":["Forming test collections with no system pooling"],"prefix":"10.1145","author":[{"given":"Mark","family":"Sanderson","sequence":"first","affiliation":[{"name":"University of Sheffield, Sheffield, UK"}]},{"given":"Hideo","family":"Joho","sequence":"additional","affiliation":[{"name":"University of Sheffield, Sheffield, UK"}]}],"member":"320","published-online":{"date-parts":[[2004,7,25]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(86)90837-8"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1008992.1009000"},{"key":"e_1_3_2_1_3_1","volume-title":"Corpora for Topic Detection and Tracking","author":"Cieri C.","year":"2002","unstructured":"Cieri , C. , Strassel , S. , Graff , D. , Martey , N. , Rennert , K. and Liberman , M .( 2002 ), Corpora for Topic Detection and Tracking , In : Allan, J.(ed.), Topic Detection and Tracking: Event-based Information Organization, 33--66, Kluwer . Cieri, C., Strassel, S., Graff, D., Martey, N., Rennert, K. and Liberman, M.(2002), Corpora for Topic Detection and Tracking, In: Allan, J.(ed.), Topic Detection and Tracking: Event-based Information Organization, 33--66, Kluwer."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.291009"},{"key":"e_1_3_2_1_5_1","volume-title":"NIST Special Publication 500-215: The 2nd Text REtrieval Conference(TREC-2)","author":"Fox E. A.","year":"1993","unstructured":"Fox , E. A. and Shaw , J. A .( 1993 ), Combination of Multiple Searches , in NIST Special Publication 500-215: The 2nd Text REtrieval Conference(TREC-2) , Gaithersburg, MD , 243--252. Fox, E. A. and Shaw, J. A.(1993), Combination of Multiple Searches, in NIST Special Publication 500-215: The 2nd Text REtrieval Conference(TREC-2), Gaithersburg, MD, 243--252."},{"key":"e_1_3_2_1_6_1","volume-title":"Proceedings of the 6th Text REtrieval Conference(TREC 6), NIST Special Publication 500-240","author":"Garofolo J. S.","year":"1997","unstructured":"Garofolo , J. S. , Voorhees , E. M. , Stanford , V. M. , Sp\u00e4rck Jones , K. (1997), TREC-6 1997 Spoken Document Retrieval Track Overview and Results , in Proceedings of the 6th Text REtrieval Conference(TREC 6), NIST Special Publication 500-240 , 83--92. Garofolo, J. S., Voorhees, E. M., Stanford, V. M., Sp\u00e4rck Jones, K.(1997), TREC-6 1997 Spoken Document Retrieval Track Overview and Results, in Proceedings of the 6th Text REtrieval Conference(TREC 6), NIST Special Publication 500-240, 83--92."},{"key":"e_1_3_2_1_7_1","volume-title":"Statistical bases of relevance assessment for the 'ideal' information retrieval test collection","author":"Gilbert H.","unstructured":"Gilbert , H. and Sp\u00e4rck Jones , K. (1979) , Statistical bases of relevance assessment for the 'ideal' information retrieval test collection , British Library Research and Development Report 5481, Computer Laboratory, University of Cambridge . Gilbert, H. and Sp\u00e4rck Jones, K.(1979), Statistical bases of relevance assessment for the 'ideal' information retrieval test collection, British Library Research and Development Report 5481, Computer Laboratory, University of Cambridge."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/243199.243281"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/258525.258594"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1012778807438"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/133160.133172"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/383952.384005"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/182.358466"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/258525.258544"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/383952.383961"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/860435.860481"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1108\/eb026588"},{"key":"e_1_3_2_1_18_1","volume-title":"Report on the need for and provision of an 'ideal' information retrieval test collection","author":"Sp\u00e4rck Jones K.","unstructured":"Sp\u00e4rck Jones , K. , Van Rijsbergen , C. J. (1975) , Report on the need for and provision of an 'ideal' information retrieval test collection , British Library Research and Development Report 5266, University Computer Laboratory , Cambridge. Sp\u00e4rck Jones, K., Van Rijsbergen, C. J.(1975), Report on the need for and provision of an 'ideal' information retrieval test collection, British Library Research and Development Report 5266, University Computer Laboratory, Cambridge."},{"key":"e_1_3_2_1_19_1","volume-title":"Report on a design study for the 'ideal' information retrieval test collection","author":"Sp\u00e4rck Jones K.","unstructured":"Sp\u00e4rck Jones , K. , Bates , R. G. (1977) , Report on a design study for the 'ideal' information retrieval test collection , British Library Research and Development Report 5428, Computer Laboratory, University of Cambridge . Sp\u00e4rck Jones, K., Bates, R. G.(1977), Report on a design study for the 'ideal' information retrieval test collection, British Library Research and Development Report 5428, Computer Laboratory, University of Cambridge."},{"key":"e_1_3_2_1_20_1","volume-title":"Encyclopedia of Statistical Sciences","author":"Stuart A.","unstructured":"Stuart , A. (1983) , Kendall's tau . In Kotz, S and Johnson, N. L., editors, Encyclopedia of Statistical Sciences , vol. 4 , 367--369. John Wiley and Sons . Stuart, A.(1983), Kendall's tau. In Kotz, S and Johnson, N. L., editors, Encyclopedia of Statistical Sciences, vol. 4, 367--369. John Wiley and Sons."},{"key":"e_1_3_2_1_21_1","unstructured":"Sullivan D.(2002) The Search Engine \"Perfect Page\" in Search Engine Watch accessed from http:\/\/searchenginewatch.com\/searchday\/02\/sd1104-pptest.html.  Sullivan D.(2002) The Search Engine \"Perfect Page\" in Search Engine Watch accessed from http:\/\/searchenginewatch.com\/searchday\/02\/sd1104-pptest.html."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.291017"},{"key":"e_1_3_2_1_23_1","volume-title":"Proceedings of the 7th Text REtrieval Conference(TREC-7) NIST Special Publication 500-242","author":"Voorhees E. M.","unstructured":"Voorhees , E. M. , Harman , D. (1998) Overview of the 7 th Text REtrieval Conference(TREC-7) , in Proceedings of the 7th Text REtrieval Conference(TREC-7) NIST Special Publication 500-242 , 1--24. Voorhees, E. M., Harman, D.(1998) Overview of the 7 th Text REtrieval Conference(TREC-7), in Proceedings of the 7th Text REtrieval Conference(TREC-7) NIST Special Publication 500-242, 1--24."},{"key":"e_1_3_2_1_24_1","volume-title":"D.(1999)","author":"Voorhees E. M.","unstructured":"Voorhees , E. M. , Harman , D.(1999) Overview of the 8th Text REtrieval Conference(TREC-8), in Proceedings of the 8th Text REtrieval Conference(TREC-8) NIST Special Publication 500-246, 1--24. Voorhees, E. M., Harman, D.(1999) Overview of the 8th Text REtrieval Conference(TREC-8), in Proceedings of the 8th Text REtrieval Conference(TREC-8) NIST Special Publication 500-246, 1--24."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/383952.383963"},{"key":"e_1_3_2_1_26_1","unstructured":"Voorhees E.(2002) Personal Communication.  Voorhees E.(2002) Personal Communication."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.291014"}],"event":{"name":"SIGIR04: The 27th ACM\/SIGIR International Symposium on Information Retrieval 2004","location":"Sheffield United Kingdom","acronym":"SIGIR04","sponsor":["ACM Association for Computing Machinery","SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1008992.1009001","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1008992.1009001","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T21:41:08Z","timestamp":1750282868000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1008992.1009001"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2004,7,25]]},"references-count":27,"alternative-id":["10.1145\/1008992.1009001","10.1145\/1008992"],"URL":"https:\/\/doi.org\/10.1145\/1008992.1009001","relation":{},"subject":[],"published":{"date-parts":[[2004,7,25]]},"assertion":[{"value":"2004-07-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}