{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:21:22Z","timestamp":1750306882470,"version":"3.41.0"},"reference-count":75,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2013,5,1]],"date-time":"2013-05-01T00:00:00Z","timestamp":1367366400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000097","name":"National Center for Research Resources","doi-asserted-by":"publisher","award":["UL1RR025747"],"award-info":[{"award-number":["UL1RR025747"]}],"id":[{"id":"10.13039\/100000097","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","award":["UL1TR000083"],"award-info":[{"award-number":["UL1TR000083"]}],"id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]},{"name":"College of Information Science and Technology at Drexel University"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2013,5]]},"abstract":"<jats:p>\n            With the ubiquitous production, distribution and consumption of information, today's digital environments such as the Web are increasingly large and decentralized. It is hardly possible to obtain central control over information collections and systems in these environments. Searching for information in these information spaces has brought about problems beyond traditional boundaries of information retrieval (IR) research. This article addresses one important aspect of scalability challenges facing information retrieval models and investigates a decentralized, organic view of information systems pertaining to search in large-scale networks. Drawing on observations from earlier studies, we conduct a series of experiments on decentralized searches in large-scale networked information spaces. Results show that how distributed systems interconnect is crucial to retrieval performance and scalability of searching. Particularly, in various experimental settings and retrieval tasks, we find a consistent phenomenon, namely, the\n            <jats:italic>Clustering Paradox<\/jats:italic>\n            , in which the level of network clustering (semantic overlay) imposes a scalability limit. Scalable searches are well supported by a specific, balanced level of network clustering emerging from local system interconnectivity. Departure from that level, either stronger or weaker clustering, leads to search performance degradation, which is dramatic in large-scale networks.\n          <\/jats:p>","DOI":"10.1145\/2457465.2457468","type":"journal-article","created":{"date-parts":[[2013,5,21]],"date-time":"2013-05-21T12:33:56Z","timestamp":1369139636000},"page":"1-36","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Studying the clustering paradox and scalability of search in highly distributed environments"],"prefix":"10.1145","volume":"31","author":[{"given":"Weimao","family":"Ke","sequence":"first","affiliation":[{"name":"Drexel University, Philadelphia, PA"}]},{"given":"Javed","family":"Mostafa","sequence":"additional","affiliation":[{"name":"University of North Carolina at Chapel Hill, Chapel Hill, NC"}]}],"member":"320","published-online":{"date-parts":[[2013,5,17]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.socnet.2005.01.007"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.64.046135"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1103\/RevModPhys.74.47"},{"key":"e_1_2_1_4_1","volume-title":"-L","author":"Albert R.","year":"1999","unstructured":"Albert , R. , Jeong , H. , and Barab\u00e1si , A . -L . 1999 . Internet : Diameter of the World-Wide Web. Nature 401, 6749, 130--131. Albert, R., Jeong, H., and Barab\u00e1si, A.-L. 1999. Internet: Diameter of the World-Wide Web. Nature 401, 6749, 130--131."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/383952.384007"},{"volume-title":"Modern Information Retrieval","author":"Baeza-Yates R.","key":"e_1_2_1_6_1","unstructured":"Baeza-Yates , R. and Ribeiro-Neto , B. 2004. Modern Information Retrieval . Addison Wesley Longman Publishing . Baeza-Yates, R. and Ribeiro-Neto, B. 2004. Modern Information Retrieval. Addison Wesley Longman Publishing."},{"volume-title":"Proceedings of the IEEE 23rd International Conference on Data Engineering (ICDE'07)","author":"Baeza-Yates R.","key":"e_1_2_1_7_1","unstructured":"Baeza-Yates , R. , Castillo , C. , Junqueira , F. , Plachouras , V. , and Silvestri , F . 2007. Challenges on distributed web retrieval . In Proceedings of the IEEE 23rd International Conference on Data Engineering (ICDE'07) . 6--20. Baeza-Yates, R., Castillo, C., Junqueira, F., Plachouras, V., and Silvestri, F. 2007. Challenges on distributed web retrieval. In Proceedings of the IEEE 23rd International Conference on Data Engineering (ICDE'07). 6--20."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1173299"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1026572910743"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/860435.860491"},{"key":"e_1_2_1_11_1","doi-asserted-by":"crossref","unstructured":"Bellifemine F. L. Caire G. and Greenwood D. 2007. Developing Multi-Agent Systems with JADE. Wiley Series in Agent Technology John Wiley & Sons.   Bellifemine F. L. Caire G. and Greenwood D. 2007. Developing Multi-Agent Systems with JADE. Wiley Series in Agent Technology John Wiley & Sons.","DOI":"10.1002\/9780470058411"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076049"},{"volume-title":"Survey of Text Mining: Clustering, Classification, and Retrieval","author":"Berry M. W.","key":"e_1_2_1_13_1","unstructured":"Berry , M. W. 2004. Survey of Text Mining: Clustering, Classification, and Retrieval . Springer . Berry, M. W. 2004. Survey of Text Mining: Clustering, Classification, and Retrieval. Springer."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1038\/nphys1130"},{"volume-title":"Advances in Information Retrieval","author":"Callan J.","key":"e_1_2_1_15_1","unstructured":"Callan , J. 2002. Distributed Information Retrieval . In Advances in Information Retrieval , W. Bruce Croft, Ed., The Information Retrieval Series, Vol . 7, Springer US , 127--150. Callan, J. 2002. Distributed Information Retrieval. In Advances in Information Retrieval, W. Bruce Croft, Ed., The Information Retrieval Series, Vol. 7, Springer US, 127--150."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/382979.383040"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/215206.215328"},{"volume-title":"Overview of the TREC 2009 web track. In Proceedings of the 18th Text Retrieval Conference (TREC'09)","author":"Clarke L. A.","key":"e_1_2_1_18_1","unstructured":"Clarke , Charles L. A. , Craswell , N. , and Soboroff , I . 2009 . Overview of the TREC 2009 web track. In Proceedings of the 18th Text Retrieval Conference (TREC'09) . Clarke, Charles L. A., Craswell, N., and Soboroff, I. 2009. Overview of the TREC 2009 web track. In Proceedings of the 18th Text Retrieval Conference (TREC'09)."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1059981.1059983"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/11574781_1"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1389-1286(99)00022-5"},{"key":"e_1_2_1_22_1","doi-asserted-by":"crossref","unstructured":"Dodds P. S. Muhamad R. and Watts D. J. 2003. An experimental study of search in global social networks. Science 301 5634 827--829.  Dodds P. S. Muhamad R. and Watts D. J. 2003. An experimental study of search in global social networks. Science 301 5634 827--829.","DOI":"10.1126\/science.1081058"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458469.1458477"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1096952.1096958"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/2.989932"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/312624.312684"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.290976"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/276627.276652"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1086\/225469"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/191839.191869"},{"key":"e_1_2_1_31_1","volume-title":"Lucene in Action","author":"Hatcher E.","unstructured":"Hatcher , E. , Gospodneti\u0107 , O. , and McCandless , M. 2010. Lucene in Action 2 nd Ed. Manning Publications . Hatcher, E., Gospodneti\u0107, O., and McCandless, M. 2010. Lucene in Action 2nd Ed. Manning Publications.","edition":"2"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076050"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/243199.243216"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the 3rd International Conference on the Practical Applications of Intelligent Agents and Multi-Agent Technology. H. S. Nwana and D. T. Ndumu, Eds.","author":"Huhns M. N.","year":"1998","unstructured":"Huhns , M. N. 1998 . Agent Foundations for Cooperative Information Systems . In Proceedings of the 3rd International Conference on the Practical Applications of Intelligent Agents and Multi-Agent Technology. H. S. Nwana and D. T. Ndumu, Eds. Huhns, M. N. 1998. Agent Foundations for Cooperative Information Systems. In Proceedings of the 3rd International Conference on the Practical Applications of Intelligent Agents and Multi-Agent Technology. H. S. Nwana and D. T. Ndumu, Eds."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/367211.367250"},{"key":"e_1_2_1_37_1","doi-asserted-by":"crossref","unstructured":"Jennings N. R. and Wooldridge M. J. 1998. Applications of Intelligent Agents. In Agent Technology: Foundations Applications and Markets Nicholas R. Jennings and Michael J. Wooldridge Eds. Springer. 3--28.   Jennings N. R. and Wooldridge M. J. 1998. Applications of Intelligent Agents. In Agent Technology: Foundations Applications and Markets Nicholas R. Jennings and Michael J. Wooldridge Eds. Springer. 3--28.","DOI":"10.1007\/978-3-662-03678-5_1"},{"volume-title":"Next Generation Search Engines: Advanced Models for Information Retrieval","author":"Ke W.","key":"e_1_2_1_38_1","unstructured":"Ke , W. 2012. Decentralized search and the clustering paradox in large scale information networks . In Next Generation Search Engines: Advanced Models for Information Retrieval , C. Jouis, I. Biskri, J. G. Ganascia, and M. Roux, Eds., IGI Global , 29--46. Ke, W. 2012. Decentralized search and the clustering paradox in large scale information networks. In Next Generation Search Engines: Advanced Models for Information Retrieval, C. Jouis, I. Biskri, J. G. Ganascia, and M. Roux, Eds., IGI Global, 29--46."},{"volume-title":"Proceedings of the 7th Workshop on Large-Scale Distributed Systems for Information Retrieval, Colocated with ACM SIGIR'09","author":"Ke W.","key":"e_1_2_1_39_1","unstructured":"Ke , W. and Mostafa , J . 2009. Strong ties vs. weak ties: Studying the clustering paradox for decentralized search . In Proceedings of the 7th Workshop on Large-Scale Distributed Systems for Information Retrieval, Colocated with ACM SIGIR'09 . 49--56. Ke, W. and Mostafa, J. 2009. Strong ties vs. weak ties: Studying the clustering paradox for decentralized search. In Proceedings of the 7th Workshop on Large-Scale Distributed Systems for Information Retrieval, Colocated with ACM SIGIR'09. 49--56."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835449.1835465"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1571941.1571947"},{"key":"e_1_2_1_42_1","volume-title":"Proceedings of the International Congress of Mathematicians.","author":"Kleinberg J.","year":"2006","unstructured":"Kleinberg , J. 2006 a. Complex networks and decentralized search algorithms . In Proceedings of the International Congress of Mathematicians. Kleinberg, J. 2006a. Complex networks and decentralized search algorithms. In Proceedings of the International Congress of Mathematicians."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148172"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1038\/35022643"},{"key":"e_1_2_1_45_1","doi-asserted-by":"crossref","unstructured":"Kleinberg J. M. Kumar R. Raghavan P. Rajagopalan S. and \n      Tomkins A. S\n  . \n  1999\n  . The Web as a graph: Measurements models and methods. In Proceedings of the 5th Annual International Conference on \n  Computing and Combinatorics Lecture Notes in Computer Science vol. \n  1627\n  . 1--17.   Kleinberg J. M. Kumar R. Raghavan P. Rajagopalan S. and Tomkins A. S. 1999. The Web as a graph: Measurements models and methods. In Proceedings of the 5th Annual International Conference on Computing and Combinatorics Lecture Notes in Computer Science vol. 1627. 1--17.","DOI":"10.1007\/3-540-48686-0_1"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0503018102"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148197"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273221.1273233"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148229"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/1135777.1135987"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/COMST.2005.1610546"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1183579.1183588"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/383952.384005"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.20081"},{"key":"e_1_2_1_56_1","doi-asserted-by":"crossref","unstructured":"Meng W. and Yu C. T. 2010. Advanced Metasearch Engine Technology. Morgan & Claypool Publishers.   Meng W. and Yu C. T. 2010. Advanced Metasearch Engine Technology. Morgan & Claypool Publishers.","DOI":"10.1007\/978-3-031-01843-5"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/505282.505284"},{"key":"e_1_2_1_58_1","first-page":"61","article-title":"Small-world","volume":"1","author":"Milgram S.","year":"1967","unstructured":"Milgram , S. 1967 . Small-world Problem. Psych. Today 1 , 1, 61 -- 67 . Milgram, S. 1967. Small-world Problem. Psych. Today 1, 1, 61--67.","journal-title":"Problem. Psych. Today"},{"key":"e_1_2_1_59_1","unstructured":"Page L. Brin S. Motwani R. and Winograd T. 1998. The PageRank citation ranking: Bringing order to the Web. Tech. rep. Stanford Digital Library Technologies Project.  Page L. Brin S. Motwani R. and Winograd T. 1998. The PageRank citation ranking: Bringing order to the Web. Tech. rep. Stanford Digital Library Technologies Project."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/944012.944016"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000010"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277827"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/860435.860490"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/944012.944017"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076051"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0800497105"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/367211.367255"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277857"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/863955.863976"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-009-9094-z"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1108\/eb026557"},{"key":"e_1_2_1_72_1","volume-title":"Six Degrees: The Science of a Connected Age","author":"Watts D.","year":"2003","unstructured":"Watts , D. 2003 . Six Degrees: The Science of a Connected Age . W.W. Norton , New York . Watts, D. 2003. Six Degrees: The Science of a Connected Age. W.W. Norton, New York."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1038\/30918"},{"key":"e_1_2_1_74_1","doi-asserted-by":"crossref","unstructured":"Watts D. J. Dodds P. S. and Newman M. E. J. 2002. Identity and search in social networks. Science 296 5571 1302--1305.  Watts D. J. Dodds P. S. and Newman M. E. J. 2002. Identity and search in social networks. Science 296 5571 1302--1305.","DOI":"10.1126\/science.1070120"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/312624.312687"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/860575.860587"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2457465.2457468","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2457465.2457468","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T08:18:36Z","timestamp":1750234716000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2457465.2457468"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,5]]},"references-count":75,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2013,5]]}},"alternative-id":["10.1145\/2457465.2457468"],"URL":"https:\/\/doi.org\/10.1145\/2457465.2457468","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"type":"print","value":"1046-8188"},{"type":"electronic","value":"1558-2868"}],"subject":[],"published":{"date-parts":[[2013,5]]},"assertion":[{"value":"2011-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-05-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}