{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T04:23:20Z","timestamp":1784262200638,"version":"3.55.0"},"reference-count":21,"publisher":"Emerald","issue":"4","license":[{"start":{"date-parts":[[2011,8,9]],"date-time":"2011-08-09T00:00:00Z","timestamp":1312848000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2011,8,9]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-heading\">Purpose<\/jats:title><jats:p>The main purpose of this study is to evaluate the effectiveness of relevance ranking on Google by comparing the system's assessment of relevance with the users' views. The research aims to find out whether the presumably objective relevance ranking of Google based on the PageRank and some other factors in fact matches users' subjective judgments of relevance.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Design\/methodology\/approach<\/jats:title><jats:p>This research investigated the relevance ranking of Google's retrieved results using 34 searches conducted by users in real search sessions. The results pages 1\u20104 (i.e. the first 40 results) were examined by the users to identify relevant documents. Based on these data the frequency of relevant documents according to the appearance order of retrieved documents in the first four results pages was calculated. The four results pages were also compared in terms of precision.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Findings<\/jats:title><jats:p>In 50 per cent and 47.06 per cent of the searches the documents ranked 5th and 1st, (i.e. from the first pages of the retrieved results) respectively, were most relevant according to the users' viewpoints. Yet even in the fourth results pages there were three documents that were judged most relevant by the users in more than 40 per cent of the searches. There were no significant differences between the precision of the four results pages except between pages 1 and 3.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Practical implications<\/jats:title><jats:p>The results will help users of search engines, especially Google, to decide how many pages of the retrieved results to examine.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Originality\/value<\/jats:title><jats:p>Search engine design will benefit from the results of this study as it experimentally evaluates the effectiveness of Google's relevance ranking.<\/jats:p><\/jats:sec>","DOI":"10.1108\/14684521111161954","type":"journal-article","created":{"date-parts":[[2011,8,20]],"date-time":"2011-08-20T07:01:52Z","timestamp":1313823712000},"page":"598-610","source":"Crossref","is-referenced-by-count":20,"title":["Relevance ranking on Google"],"prefix":"10.1108","volume":"35","author":[{"given":"Nadjla","family":"Hariri","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"140","reference":[{"key":"key2022031220131925500_b1","doi-asserted-by":"crossref","unstructured":"Agrahri, A.K., Manickam, D.A.T. and Riedl, J. (2008), \u201cCan people collaborate to improve the relevance of search results?\u201d, Proceedings of the 2008 ACM Conference on Recommender Systems, Lausanne, ACM Press, New York, NY, pp. 283\u20106.","DOI":"10.1145\/1454008.1454052"},{"key":"key2022031220131925500_b2","doi-asserted-by":"crossref","unstructured":"Bade, D. (2007), \u201cRelevance ranking is not relevance ranking or, when the user is not the user, the search results are not search results\u201d, Online Information Review, Vol. 31 No. 6, pp. 831\u201044.","DOI":"10.1108\/14684520710841793"},{"key":"key2022031220131925500_b3","doi-asserted-by":"crossref","unstructured":"Bar\u2010Ilan, J., Mat\u2010Hassan, M. and Levene, M. (2006a), \u201cMethods for comparing rankings of search engine results\u201d, Computer Networks, Vol. 50 No. 10, pp. 1448\u201063.","DOI":"10.1016\/j.comnet.2005.10.020"},{"key":"key2022031220131925500_b4","doi-asserted-by":"crossref","unstructured":"Bar\u2010Ilan, J., Levene, M. and Mat\u2010Hassan, M. (2006b), \u201cMethods for evaluating dynamic changes in search engine rankings: a case study\u201d, Journal of Documentation, Vol. 62 No. 6, pp. 708\u201029.","DOI":"10.1108\/00220410610714930"},{"key":"key2022031220131925500_b5","doi-asserted-by":"crossref","unstructured":"Bar\u2010Ilan, J., Keenoy, K., Yaari, E. and Levene, M. (2007), \u201cUser rankings of search engine results\u201d, Journal of the American Society for Information Science and Technology, Vol. 58 No. 9, pp. 1254\u201066.","DOI":"10.1002\/asi.20608"},{"key":"key2022031220131925500_b6","doi-asserted-by":"crossref","unstructured":"Chowdhury, A. and Soboroff, I. (2002), \u201cAutomatic evaluation of world wide web search services\u201d, Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Tampere, Finland, ACM Press, New York, NY, pp. 421\u20102.","DOI":"10.1145\/564376.564474"},{"key":"key2022031220131925500_b7","unstructured":"comScore (2010), \u201ccomScore reports global search market growth of 46 per cent in 2009\u201d, available at: www.comscore.com\/Press_Events\/Press_Releases\/2010\/1\/Global_Search_Market_Grows_46_Per cent_in_2009 (accessed 19 April 2011)."},{"key":"key2022031220131925500_b8","unstructured":"Google (2010), \u201cAbout Google, corporate information, technology overview\u201d, available at: www.google.com\/intl\/en\/corporate\/tech.html (accessed 4 September 2010)."},{"key":"key2022031220131925500_b9","doi-asserted-by":"crossref","unstructured":"Hariri, N. (2008), \u201cAn investigation on effectiveness of the \u2018similar pages\u2019 feature of Google\u201d, Online Information Review, Vol. 32 No. 3, pp. 370\u20108.","DOI":"10.1108\/14684520810889673"},{"key":"key2022031220131925500_b10","doi-asserted-by":"crossref","unstructured":"Hawking, D., Craswell, N., Thistlewaite, P. and Harman, D. (1999), \u201cResults and challenges in web search evaluation\u201d, Computer Networks, Vol. 31 Nos 11\u201016, pp. 1321\u201030.","DOI":"10.1016\/S1389-1286(99)00024-9"},{"key":"key2022031220131925500_b11","doi-asserted-by":"crossref","unstructured":"Jansen, B.J. and Spink, A. (2005), \u201cAn analysis of web searching by European AlltheWeb.com users\u201d, Information Processing and Management, Vol. 41, pp. 361\u201081.","DOI":"10.1016\/S0306-4573(03)00067-0"},{"key":"key2022031220131925500_b12","unstructured":"Jesdanun, A. (2008), \u201cGoogle takes lead as most popular US site. Search giant edges past Yahoo! for having most unique visitors in April\u201d, Associated Press, available at: www.msnbc.msn.com\/id\/24624874\/ (accessed 19 April 2011)."},{"key":"key2022031220131925500_b13","unstructured":"Meadow, C.T., Boyce, B.R., Kraft, D.H. and Barry, C. (2007), Text Information Retrieval Systems, Academic Press, London."},{"key":"key2022031220131925500_b14","unstructured":"Nowicki, S.A. (2002), \u201cStudents' success with worldwide web search engines: retrieving relevant results with respect to end\u2010user relevance judgments\u201d, PhD thesis, Nova Southeastern University, Fort Lauderdale, FL."},{"key":"key2022031220131925500_b15","doi-asserted-by":"crossref","unstructured":"Patil, S., Alpert, S.R., Karat, J. and Wolf, C. (2005), in Costabile, M.F. and Patern\u00f2, F. (Eds), \u201cTHAT's what I was looking for: comparing user\u2010rated relevance with search engine rankings\u201d, Human\u2010Computer Interaction, Proceedings of the INTERACT 2005 International Conference Proceedings, Lecture Notes in Computer Science, Vol. 3855, Springer, Berlin, pp. 117\u201029.","DOI":"10.1007\/11555261_13"},{"key":"key2022031220131925500_b16","unstructured":"Sherman, C. (2005), \u201cA new f\u2010word for Google search results\u201d, available at: http:\/\/searchenginewatch.com\/3488076 (accessed 19 April 2011)."},{"key":"key2022031220131925500_b17","doi-asserted-by":"crossref","unstructured":"Silverstein, C., Henzinger, M., Marais, H. and Moricz, M. (1999), \u201cAnalysis of a very large web search engine query log\u201d, ACM SIGIR Forum, available at: www.acm.org\/sigir\/forum\/F99\/Silverstein.pdf (accessed 20 April 2011).","DOI":"10.1145\/331403.331405"},{"key":"key2022031220131925500_b18","doi-asserted-by":"crossref","unstructured":"Singhal, A. and Kaszkiel, M. (2001), \u201cA case study in web search using TREC algorithms\u201d, Proceedings of the 10th International Worldwide Web Conference, May, available at: www10.org\/cdrom\/papers\/pdf\/p317.pdf (accessed 20 April 2011).","DOI":"10.1145\/371920.372186"},{"key":"key2022031220131925500_b19","doi-asserted-by":"crossref","unstructured":"Spink, A., Ozmutlu, S., Ozmutlu, H.C. and Jansen, B.J. (2002), \u201cUS versus European web searching trends\u201d, SIGIR Forum,Vol. 36 No. 2, pp. 32\u20108, available at: www.acm.org\/sigir\/forum\/F2002\/spink.pdf (accessed 3 November 2008).","DOI":"10.1145\/792550.792555"},{"key":"key2022031220131925500_b20","doi-asserted-by":"crossref","unstructured":"Vaughan, L. (2004), \u201cNew measurements for search engine evaluation proposed and tested\u201d, Information Processing and Management, Vol. 40 No. 4, pp. 677\u201091.","DOI":"10.1016\/S0306-4573(03)00043-8"},{"key":"key2022031220131925500_b21","unstructured":"Veronis, J. (2006), \u201cA comparative study of six search engines\u201d, available at: www.up.univ\u2010mrs.fr\/veronis\/pdf\/2006\u2010comparative\u2010study.pdf (accessed 20 April 2011)."}],"container-title":["Online Information Review"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/14684521111161954","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/14684521111161954\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/14684521111161954\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T00:41:43Z","timestamp":1753404103000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/oir\/article\/35\/4\/598-610\/447310"}},"subtitle":["Are top ranked results really considered more relevant by the users?"],"short-title":[],"issued":{"date-parts":[[2011,8,9]]},"references-count":21,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2011,8,9]]}},"alternative-id":["10.1108\/14684521111161954"],"URL":"https:\/\/doi.org\/10.1108\/14684521111161954","relation":{},"ISSN":["1468-4527"],"issn-type":[{"value":"1468-4527","type":"print"}],"subject":[],"published":{"date-parts":[[2011,8,9]]}}}