{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T06:11:46Z","timestamp":1775283106236,"version":"3.50.1"},"reference-count":77,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2009,5,1]],"date-time":"2009-05-01T00:00:00Z","timestamp":1241136000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2009,5]]},"abstract":"<jats:p>In federated information retrieval, a query is routed to multiple collections and a single answer list is constructed by combining the results. Such metasearch provides a mechanism for locating documents on the hidden Web and, by use of sampling, can proceed even when the collections are uncooperative. However, the similarity scores for documents returned from different collections are not comparable, and, in uncooperative environments, document scores are unlikely to be reported. We introduce a new merging method for uncooperative environments, in which similarity scores for the sampled documents held for each collection are used to estimate global scores for the documents returned per query. This method requires no assumptions about properties such as the retrieval models used. Using experiments on a wide range of collections, we show that in many cases our merging methods are significantly more effective than previous techniques.<\/jats:p>","DOI":"10.1145\/1508850.1508852","type":"journal-article","created":{"date-parts":[[2009,5,19]],"date-time":"2009-05-19T16:47:42Z","timestamp":1242751662000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":38,"title":["Robust result merging using sample-based score estimates"],"prefix":"10.1145","volume":"27","author":[{"given":"Milad","family":"Shokouhi","sequence":"first","affiliation":[{"name":"RMIT University"}]},{"given":"Justin","family":"Zobel","sequence":"additional","affiliation":[{"name":"RMIT University"}]}],"member":"320","published-online":{"date-parts":[[2009,5,19]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the International Conference on Information Technology: Coding and Computing. IEEE Computer Society Press","author":"Abbaci F.","unstructured":"Abbaci , F. , Savoy , J. , and Beigbeder , M . 2002. A methodology for collection selection in heterogeneous contexts . In Proceedings of the International Conference on Information Technology: Coding and Computing. IEEE Computer Society Press , Los Alamitos, CA, 529. Abbaci, F., Savoy, J., and Beigbeder, M. 2002. A methodology for collection selection in heterogeneous contexts. In Proceedings of the International Conference on Information Technology: Coding and Computing. IEEE Computer Society Press, Los Alamitos, CA, 529."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/956863.956953"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/383952.384007"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.v57:3"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148277"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/11880561_26"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1135777.1135833"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/11880561_10"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0169-7552(98)00127-5"},{"key":"e_1_2_1_10_1","first-page":"127","article-title":"Distributed information retrieval. Advances in Information Retrieval. Kluwer, Norwell, MA","volume":"5","author":"Callan J.","year":"2000","unstructured":"Callan , J. 2000 . Distributed information retrieval. Advances in Information Retrieval. Kluwer, Norwell, MA , Chapter 5 , 127 -- 150 . Callan, J. 2000. Distributed information retrieval. Advances in Information Retrieval. Kluwer, Norwell, MA, Chapter 5, 127--150.","journal-title":"Chapter"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/382979.383040"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/304182.304224"},{"key":"e_1_2_1_13_1","unstructured":"Callan J. Croft B. and Broglio J. 1997. TREC and TIPSTER experiments with INQUERY. In Readings in Information Retrieval. Morgan Kaufmann San Francisco CA 436--439.   Callan J. Croft B. and Broglio J. 1997. TREC and TIPSTER experiments with INQUERY. In Readings in Information Retrieval. Morgan Kaufmann San Francisco CA 436--439."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/215206.215328"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148230"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/336597.336628"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 10th Australasian Database Conference. Springer-Verlag","author":"Craswell N.","unstructured":"Craswell , N. , Hawking , D. , and Thistlewaite , P . 1999. Merging results from isolated search engines . In Proceedings of the 10th Australasian Database Conference. Springer-Verlag , Auckland, New Zealand, 189--200. Craswell, N., Hawking, D., and Thistlewaite, P. 1999. Merging results from isolated search engines. In Proceedings of the 10th Australasian Database Conference. Springer-Verlag, Auckland, New Zealand, 189--200."},{"key":"e_1_2_1_18_1","first-page":"1","article-title":"Combining approaches to information retrieval. Advances in Information Retrieval. Kluwer, Norwell, MA","volume":"1","author":"Croft B.","year":"2000","unstructured":"Croft , B. 2000 . Combining approaches to information retrieval. Advances in Information Retrieval. Kluwer, Norwell, MA , Chapter 1 , 1 -- 36 . Croft, B. 2000. Combining approaches to information retrieval. Advances in Information Retrieval. Kluwer, Norwell, MA, Chapter 1, 1--36.","journal-title":"Chapter"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/256163.256164"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the Second International Symposium on Cooperative Database Systems for Advanced Applications (CODAS'99)","author":"D'Souza D.","unstructured":"D'Souza , D. and Thom , J . 1999. Collection selection using n-term indexing . In Proceedings of the Second International Symposium on Cooperative Database Systems for Advanced Applications (CODAS'99) . Springer, Wollongong, Australia, 52--63. D'Souza, D. and Thom, J. 1999. Collection selection using n-term indexing. In Proceedings of the Second International Symposium on Cooperative Database Systems for Advanced Applications (CODAS'99). Springer, Wollongong, Australia, 52--63."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(03)00008-6"},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the Second Text REtrieval Conference. NIST Special Publication. National Institute of Science and Technology","author":"Fox E.","unstructured":"Fox , E. and Shaw , J . 1993. Combination of multiple searches . In Proceedings of the Second Text REtrieval Conference. NIST Special Publication. National Institute of Science and Technology , Gaithersburg, MD, 243--252. Fox, E. and Shaw, J. 1993. Combination of multiple searches. In Proceedings of the Second Text REtrieval Conference. NIST Special Publication. National Institute of Science and Technology, Gaithersburg, MD, 243--252."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/312624.312684"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/314516.314517"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the 27th Australasian Computer Science Conference","author":"Garcia S.","unstructured":"Garcia , S. , Williams , H. , and Cannane , A . 2004. Access-ordered indexes . In Proceedings of the 27th Australasian Computer Science Conference ( Darlinghurst, Australia). 7--14. Garcia, S., Williams, H., and Cannane, A. 2004. Access-ordered indexes. In Proceedings of the 27th Australasian Computer Science Conference (Darlinghurst, Australia). 7--14."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/319950.319980"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/253260.253299"},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of the 21st International Conference on Very Large Data Bases","author":"Gravano L.","unstructured":"Gravano , L. and Garcia-Molina , H . 1995. Generalizing GlOSS to vector-space databases and broker hierarchies . In Proceedings of the 21st International Conference on Very Large Data Bases ( San Francisco, CA). 78--89. Gravano, L. and Garcia-Molina, H. 1995. Generalizing GlOSS to vector-space databases and broker hierarchies. In Proceedings of the 21st International Conference on Very Large Data Bases (San Francisco, CA). 78--89."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/191839.191869"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the Third International Conference on Parallel and Distributed Information Systems","author":"Gravano L.","unstructured":"Gravano , L. , Garcia-Molina , H. , and Tomasic , A . 1994b. Precision and recall of GlOSS estimators for database discovery . In Proceedings of the Third International Conference on Parallel and Distributed Information Systems ( Washington, DC). 103--106. Gravano, L., Garcia-Molina, H., and Tomasic, A. 1994b. Precision and recall of GlOSS estimators for database discovery. In Proceedings of the Third International Conference on Parallel and Distributed Information Systems (Washington, DC). 103--106."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/320248.320252"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/635484.635485"},{"key":"e_1_2_1_33_1","volume-title":"Linear Regression","author":"Gross J.","unstructured":"Gross , J. 2003. Linear Regression . Springer , Berlin, Germany . Gross, J. 2003. Linear Regression. Springer, Berlin, Germany."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1031453.1031456"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1031453.1031456"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1007568.1007655"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076063"},{"key":"e_1_2_1_38_1","first-page":"659","article-title":"Document retrieval over networks wherein ranking and relevance scores are computed at the client for multiple database documents","volume":"5","author":"Kirsch T.","year":"2003","unstructured":"Kirsch , T. 2003 . Document retrieval over networks wherein ranking and relevance scores are computed at the client for multiple database documents . U.S. Patent 5 , 659 ,732. Kirsch, T. 2003. Document retrieval over networks wherein ranking and relevance scores are computed at the client for multiple database documents. U.S. Patent 5,659,732.","journal-title":"U.S. Patent"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/383952.383970"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/354756.354830"},{"key":"e_1_2_1_41_1","volume-title":"Proceedings of the 7th International Conference on the World Wide Web. Elsevier Science Publishers B. V.","author":"Lawrence S.","unstructured":"Lawrence , S. and Giles , C . 1998. Inquirus, the NECi meta search engine . In Proceedings of the 7th International Conference on the World Wide Web. Elsevier Science Publishers B. V. , Brisbane, Australia, 95--105. Lawrence, S. and Giles, C. 1998. Inquirus, the NECi meta search engine. In Proceedings of the 7th International Conference on the World Wide Web. Elsevier Science Publishers B. V., Brisbane, Australia, 95--105."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/258525.258587"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148197"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/584792.584847"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/383952.384005"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/860435.860489"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/502585.502617"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/511446.511490"},{"key":"e_1_2_1_50_1","volume-title":"Proceedings of the Euorpean Conference on Information Retrieval. Springer","author":"Paltoglou G.","unstructured":"Paltoglou , G. , Salampasis , M. , and Satratzemi , M . 2007. Results merging algorithm using multiple regression models . In Proceedings of the Euorpean Conference on Information Retrieval. Springer , Rome, Italy, 173--184. Paltoglou, G., Salampasis, M., and Satratzemi, M. 2007. Results merging algorithm using multiple regression models. In Proceedings of the Euorpean Conference on Information Retrieval. Springer, Rome, Italy, 173--184."},{"key":"e_1_2_1_51_1","volume-title":"Readings in Information Retrieval. Morgan Kaufmann","author":"Porter M.","unstructured":"Porter , M. 1997. An algorithm for suffix stripping . In Readings in Information Retrieval. Morgan Kaufmann , San Francisco, CA , 313--316. Porter, M. 1997. An algorithm for suffix stripping. In Readings in Information Retrieval. Morgan Kaufmann, San Francisco, CA, 313--316."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/944012.944016"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/502585.502618"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(02)00122-X"},{"key":"e_1_2_1_55_1","volume-title":"Proceedings of the 4th International Conference on the World Wide Web. Oreilly","author":"Selberg E.","unstructured":"Selberg , E. and Etzioni , O . 1995. Multi-service search and comparison using the metacrawler . In Proceedings of the 4th International Conference on the World Wide Web. Oreilly , Boston, MA. Selberg, E. and Etzioni, O. 1995. Multi-service search and comparison using the metacrawler. In Proceedings of the 4th International Conference on the World Wide Web. Oreilly, Boston, MA."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/64.577468"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.5555\/1763653.1763674"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1007\/11610113_7"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277827"},{"key":"e_1_2_1_60_1","volume-title":"Proceedings of the 18th Australasian Database Conference. CRPIT","volume":"63","author":"Shokouhi M.","unstructured":"Shokouhi , M. , Zobel , J. , and Bernstein , Y . 2007. Distributed text retrieval from overlapping collections . In Proceedings of the 18th Australasian Database Conference. CRPIT , vol. 63 . ACS, Ballarat, Australia, 141--150. Shokouhi, M., Zobel, J., and Bernstein, Y. 2007. Distributed text retrieval from overlapping collections. In Proceedings of the 18th Australasian Database Conference. CRPIT, vol. 63. ACS, Ballarat, Australia, 141--150."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148227"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/564376.564382"},{"key":"e_1_2_1_63_1","volume-title":"Proeedings of the SIGIR 2003 Workshop on Distributed Information Retrieval","author":"Si L.","unstructured":"Si , L. and Callan , J . 2003a. The effect of database size distribution on resource selection algorithms . In Proeedings of the SIGIR 2003 Workshop on Distributed Information Retrieval ( Toronto, Ont., Canada). 31--42. Si, L. and Callan, J. 2003a. The effect of database size distribution on resource selection algorithms. In Proeedings of the SIGIR 2003 Workshop on Distributed Information Retrieval (Toronto, Ont., Canada). 31--42."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/860435.860490"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/944012.944017"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/1031171.1031180"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076051"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/584792.584856"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277828"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1009980820262"},{"key":"e_1_2_1_72_1","volume-title":"Proceedings of the 30th International Conference on Very Large Data Bases","author":"Wang Y.","year":"2004","unstructured":"Wang , Y. and DeWitt , D. 2004 . Computing PageRank in a distributed internet search engine system . In Proceedings of the 30th International Conference on Very Large Data Bases ( Toronto, Ont., Canada). 420--431. Wang, Y. and DeWitt, D. 2004. Computing PageRank in a distributed internet search engine system. In Proceedings of the 30th International Conference on Very Large Data Bases (Toronto, Ont., Canada). 420--431."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2005.08.004"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-007-9023-y"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.290974"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/312624.312687"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277910"},{"key":"e_1_2_1_78_1","volume-title":"Notes on the lemur TFIDF model. School of Computer Science","author":"Zhai C.","unstructured":"Zhai , C. 2001. Notes on the lemur TFIDF model. School of Computer Science . Carnegie Mellon University, Pittsburgh , PA. unpublished report. www.cs.cmu.edu\/~lemur\/1.1\/tfidf.ps. Zhai, C. 2001. Notes on the lemur TFIDF model. School of Computer Science. Carnegie Mellon University, Pittsburgh, PA. unpublished report. www.cs.cmu.edu\/~lemur\/1.1\/tfidf.ps."},{"key":"e_1_2_1_79_1","volume-title":"Proceedings of the Australian Document Computing Symposium","author":"Zobel J.","year":"1997","unstructured":"Zobel , J. 1997 . Collection selection via lexicon inspection . In Proceedings of the Australian Document Computing Symposium ( Melbourne, Australia). 74--80. Zobel, J. 1997. Collection selection via lexicon inspection. In Proceedings of the Australian Document Computing Symposium (Melbourne, Australia). 74--80."}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1508850.1508852","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1508850.1508852","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T13:29:42Z","timestamp":1750253382000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1508850.1508852"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2009,5]]},"references-count":77,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2009,5]]}},"alternative-id":["10.1145\/1508850.1508852"],"URL":"https:\/\/doi.org\/10.1145\/1508850.1508852","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2009,5]]},"assertion":[{"value":"2006-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2008-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2009-05-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}