{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,29]],"date-time":"2025-12-29T13:50:20Z","timestamp":1767016220818,"version":"3.41.2"},"reference-count":23,"publisher":"Emerald","issue":"6","license":[{"start":{"date-parts":[[2006,11,1]],"date-time":"2006-11-01T00:00:00Z","timestamp":1162339200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2006,11,1]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-heading\">Purpose<\/jats:title><jats:p>The objective of this paper is to characterize the changes in the rankings of the top ten results of major search engines over time and to compare the rankings between these engines.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Design\/methodology\/approach<\/jats:title><jats:p>The papers compare rankings of the top\u2010ten results of the search engines Google and AlltheWeb on ten identical queries over a period of three weeks. Only the top\u2010ten results were considered, since users do not normally inspect more than the first results page returned by a search engine. The experiment was repeated twice, in October 2003 and in January 2004, in order to assess changes to the top\u2010ten results of some of the queries during the three months interval. In order to assess the changes in the rankings, three measures were computed for each data collection point and each search engine.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Findings<\/jats:title><jats:p>The findings in this paper show that the rankings of AlltheWeb were highly stable over each period, while the rankings of Google underwent constant yet minor changes, with occasional major ones. Changes over time can be explained by the dynamic nature of the web or by fluctuations in the search engines' indexes. The top\u2010ten results of the two search engines had surprisingly low overlap. With such small overlap, the task of comparing the rankings of the two engines becomes extremely challenging.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Originality\/value<\/jats:title><jats:p>The paper shows that because of the abundance of information on the web, ranking search results is of extreme importance. The paper compares several measures for computing the similarity between rankings of search tools, and shows that none of the measures is fully satisfactory as a standalone measure. It also demonstrates the apparent differences in the ranking algorithms of two widely used search engines.<\/jats:p><\/jats:sec>","DOI":"10.1108\/00220410610714930","type":"journal-article","created":{"date-parts":[[2007,1,15]],"date-time":"2007-01-15T09:07:03Z","timestamp":1168852023000},"page":"708-729","source":"Crossref","is-referenced-by-count":13,"title":["Methods for evaluating dynamic changes in search engine rankings: a case study"],"prefix":"10.1108","volume":"62","author":[{"given":"Judit","family":"Bar\u2010Ilan","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mark","family":"Levene","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mazlita","family":"Mat\u2010Hassan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","reference":[{"unstructured":"AlltheWeb (2004), available at: www.alltheweb.com (accessed February 18, 2004).","key":"key2022030719495366700_b1"},{"doi-asserted-by":"crossref","unstructured":"Chowdhury, A. and Soboroff, I. (2002), \u201cAutomatic evaluation of worldwide web search services\u201d, Proceedings of the 25th Annual International ACM SIGIR Conference, pp. 421\u20102.","key":"key2022030719495366700_b2","DOI":"10.1145\/564376.564474"},{"doi-asserted-by":"crossref","unstructured":"Diaconis, P. and Graham, R.L. (1977), \u201cSpearman's footrule as a measure of disarray\u201d, Journal of the Royal Statistical Society, Vol. 39 No. 2, pp. 262\u20108 (Series B (Methodological)).","key":"key2022030719495366700_b3","DOI":"10.1111\/j.2517-6161.1977.tb01624.x"},{"doi-asserted-by":"crossref","unstructured":"Dwork, C., Kumar, R., Naor, M. and Sivakumar, D. (2001), \u201cRank aggregation methods for the web\u201d, Proceedings of the 10th Worldwide Web Conference, May, Hong\u2010Kong, pp. 613\u201022.","key":"key2022030719495366700_b4","DOI":"10.1145\/371920.372165"},{"doi-asserted-by":"crossref","unstructured":"Fagin, R., Kumar, R. and Sivakumar, D. (2003), \u201cComparing top k lists\u201d, SIAM Journal on Discrete Mathematics, Vol. 17 No. 1, pp. 134\u201060.","key":"key2022030719495366700_b5","DOI":"10.1137\/S0895480102412856"},{"unstructured":"Google (2003a), Google Information for Webmasters, available at: www.google.com\/webmasters\/seo.html (accessed February 18, 2004).","key":"key2022030719495366700_b6"},{"unstructured":"Google (2003b), Google Information for Webmasters, available at: www.google.com\/webmasters\/4.html (accessed February 18, 2004).","key":"key2022030719495366700_b7"},{"unstructured":"Google (2004), Google, available at: www.google.com (accessed February 18, 2004).","key":"key2022030719495366700_b8"},{"doi-asserted-by":"crossref","unstructured":"Hawking, D., Craswell, N., Bailey, P. and Griffiths, K. (2001), \u201cMeasuring search engine quality\u201d, Information Retrieval, Vol. 4, pp. 33\u201059.","key":"key2022030719495366700_b9","DOI":"10.1023\/A:1011468107287"},{"unstructured":"Nielsen\/NetRatings (2004), NetView Usage Metrics, available at: www.netratings.com\/news.jsp?section=dat_to (accessed September 16, 2004).","key":"key2022030719495366700_b11"},{"unstructured":"Price, G. (2004), \u201cGoogle ups total page count\u201d, Resourceshelf, available at: http:\/\/www.resourceshelf.com\/archives\/2004_02_01_resourceshelf_archive.html#107702946623981034 (accessed February 18).","key":"key2022030719495366700_b12"},{"doi-asserted-by":"crossref","unstructured":"Silverstein, C., Henzinger, M., Marais, H. and Moricz, M. (1999), \u201cAnalysis of a very large web search engine query log\u201d, ACM SIGIR Forum, Vol. 33 No. 1, available at: www.acm.org\/sigir\/forum\/F99\/Silverstein.pdf (accessed February 18, 2004).","key":"key2022030719495366700_b13","DOI":"10.1145\/331403.331405"},{"doi-asserted-by":"crossref","unstructured":"Singhal, A. and Kaszkiel, M. (2001), \u201cA case study in web search using TREC algorithms\u201d, Proceedings of the 10th International Worldwide Web Conference, May, pp. 708\u201016, available at: www10.org\/cdrom\/papers\/pdf\/p317.pdf (accessed February 18, 2004).","key":"key2022030719495366700_b14","DOI":"10.1145\/371920.372186"},{"doi-asserted-by":"crossref","unstructured":"Spink, A., Ozmutlu, S., Ozmutlu, H.C. and Jansen, B.J. (2002), \u201cUS versus European web searching trends\u201d, SIGIR Forum, Fall, available at: www.acm.org\/sigir\/forum\/F2002\/spink.pdf (accessed February 18, 2004).","key":"key2022030719495366700_b15","DOI":"10.1145\/792550.792555"},{"doi-asserted-by":"crossref","unstructured":"Soboroff, I., Nicholas, C. and Cahan, P. (2001), \u201cRanking retrieval systems without relevance judgments\u201d, Proceedings of the 24th Annual International ACM SIGIR Conference, pp. 66\u201072.","key":"key2022030719495366700_b16","DOI":"10.1145\/383952.383961"},{"unstructured":"Su, L.T., Chen, H.L. and Dong, X.Y. (1998), \u201cEvaluation of web\u2010based search engines from the end\u2010user's perspective: a pilot study\u201d, Proceedings of the ASIS Annual Meeting, Vol. 35, pp. 348\u201061.","key":"key2022030719495366700_b17"},{"unstructured":"Sullivan, D. (2003a), Yahoo to Buy Overture, available at: http:\/\/searchenginewatch.com\/searchday\/article.php\/2234821 (accessed October 4, 2004).","key":"key2022030719495366700_b18"},{"unstructured":"Sullivan, D. (2003b), Buying Your Way in: Search Engine Advertising Chart, available at: www.searchenginewatch.com\/webmasters\/article.php\/2167941 (accessed February 18, 2004).","key":"key2022030719495366700_b19"},{"unstructured":"Sullivan, D. (2003c), Florida Google Dance Resources, available at: www.searchenginewatch.com\/searchday\/article.php\/3285661 (accessed February 18, 2004).","key":"key2022030719495366700_b20"},{"unstructured":"Sullivan, D. and Sherman, C. (2004), The 4th Annual Search Engine Watch 2003 Awards, available at: www.searchenginewatch.com\/awards\/article.php\/3309841 (accessed February 18, 2004).","key":"key2022030719495366700_b21"},{"doi-asserted-by":"crossref","unstructured":"Vaughan, L. (2004), \u201cNew measurements for search engine evaluation proposed and tested\u201d, Information Processing and Management, Vol. 40, pp. 677\u201091.","key":"key2022030719495366700_b22","DOI":"10.1016\/S0306-4573(03)00043-8"},{"doi-asserted-by":"crossref","unstructured":"Voorhees, E.M. (2000), \u201cVariations in relevance judgments and the measurement of retrieval effectiveness\u201d, Information Processing and Management, Vol. 36, pp. 697\u2010716.","key":"key2022030719495366700_b23","DOI":"10.1016\/S0306-4573(00)00010-8"},{"doi-asserted-by":"crossref","unstructured":"Hawking, D., Craswell, N., Thistlewaite, P. and Harman, D. (2004), \u201cResults and challenges in web search evaluation\u201d, Computer Networks, Vol. 31 Nos 11\u201016, pp. 1321\u201030.","key":"key2022030719495366700_frg1","DOI":"10.1016\/S1389-1286(99)00024-9"}],"container-title":["Journal of Documentation"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/00220410610714930","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/00220410610714930\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/00220410610714930\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T23:37:44Z","timestamp":1753400264000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/jd\/article\/62\/6\/708-729\/433904"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2006,11,1]]},"references-count":23,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2006,11,1]]}},"alternative-id":["10.1108\/00220410610714930"],"URL":"https:\/\/doi.org\/10.1108\/00220410610714930","relation":{},"ISSN":["0022-0418"],"issn-type":[{"type":"print","value":"0022-0418"}],"subject":[],"published":{"date-parts":[[2006,11,1]]}}}