{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T20:06:23Z","timestamp":1776974783437,"version":"3.51.4"},"reference-count":40,"publisher":"Emerald","issue":"6","license":[{"start":{"date-parts":[[2008,10,17]],"date-time":"2008-10-17T00:00:00Z","timestamp":1224201600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2008,10,17]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-heading\">Purpose<\/jats:title><jats:p>The purpose of this paper is to compare five major web search engines (Google, Yahoo, MSN, Ask.com, and Seekport) for their retrieval effectiveness, taking into account not only the results, but also the results descriptions.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Design\/methodology\/approach<\/jats:title><jats:p>The study uses real\u2010life queries. Results are made anonymous and are randomized. Results are judged by the persons posing the original queries.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Findings<\/jats:title><jats:p>The two major search engines, Google and Yahoo, perform best, and there are no significant differences between them. Google delivers significantly more relevant result descriptions than any other search engine. This could be one reason for users perceiving this engine as superior.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Research limitations\/implications<\/jats:title><jats:p>The study is based on a user model where the user takes into account a certain amount of results rather systematically. This may not be the case in real life.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Practical implications<\/jats:title><jats:p>The paper implies that search engines should focus on relevant descriptions. Searchers are advised to use other search engines in addition to Google.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Originality\/value<\/jats:title><jats:p>This is the first major study comparing results and descriptions systematically and proposes new retrieval measures to take into account results descriptions.<\/jats:p><\/jats:sec>","DOI":"10.1108\/00220410810912451","type":"journal-article","created":{"date-parts":[[2008,11,1]],"date-time":"2008-11-01T08:07:40Z","timestamp":1225526860000},"page":"915-937","source":"Crossref","is-referenced-by-count":68,"title":["The retrieval effectiveness of web search engines: considering results descriptions"],"prefix":"10.1108","volume":"64","author":[{"given":"Dirk","family":"Lewandowski","sequence":"first","affiliation":[]}],"member":"140","reference":[{"key":"key2022031319461130100_b1","doi-asserted-by":"crossref","unstructured":"Borlund, P. (2003), \u201cThe concept of relevance in IR\u201d, Journal of the American Society for Information Science and Technology, Vol. 54 No. 10, pp. 913\u201025.","DOI":"10.1002\/asi.10286"},{"key":"key2022031319461130100_b2","doi-asserted-by":"crossref","unstructured":"Broder, A. (2002), \u201cA taxonomy of web search\u201d, SIGIR Forum, Vol. 36 No. 2, pp. 3\u201010.","DOI":"10.1145\/792550.792552"},{"key":"key2022031319461130100_b3","doi-asserted-by":"crossref","unstructured":"Buckley, C. and Voorhees, E.M. (2000), \u201cEvaluating evaluation measure stability\u201d, Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Athens, Greece, ACM Press, New York, NY, pp. 33\u201040.","DOI":"10.1145\/345508.345543"},{"key":"key2022031319461130100_b4","unstructured":"Burns, E. (2007), \u201cUS search engine rankings, April 2007\u201d Search Engine Watch, available at: www.searchenginewatch.com\/showPage.html?page\u2009=\u20093626021."},{"key":"key2022031319461130100_b5","unstructured":"Chu, H. and Rosenthal, M. (1996), \u201cSearch engines for the world wide web: a comparative study and evaluation methodology\u201d, Proceedings of the 59th ASIS Annual Meeting, available at: www.asis.org\/annual\u201096\/ElectronicProceedings\/chu.html."},{"key":"key2022031319461130100_b6","doi-asserted-by":"crossref","unstructured":"Crystal, A. and Greenberg, J. (2006), \u201cRelevance criteria identified by health information users during web searches\u201d, Journal of the American Society for Information Science and Technology, Vol. 57 No. 10, pp. 1368\u201082.","DOI":"10.1002\/asi.20436"},{"key":"key2022031319461130100_b7","doi-asserted-by":"crossref","unstructured":"Della Mea, V. and Mizzaro, S. (2004), \u201cMeasuring retrieval effectiveness: a new proposal and a first experimental validation\u201d, Journal of the American Society for Information Science and Technology, Vol. 55 No. 6, pp. 530\u201043.","DOI":"10.1002\/asi.10408"},{"key":"key2022031319461130100_b8","unstructured":"Ding, W. and Marchionini, G. (1996), \u201cA comparative study of web search service performance\u201d, Proceedings of the 59th ASIS Annual Meeting, pp. 136\u201042."},{"key":"key2022031319461130100_b9","unstructured":"Dresel, R., H\u00f6rnig, D., Kaluza, H., Peter, A., Ro\u00dfmann, N. and Sieber, W. (2001), \u201cEvaluation deutscher Web\u2010Suchwerkzeuge\u201d, Nachrichten f\u00fcr Dokumentation, Vol. 52 No. 7, pp. 381\u201092."},{"key":"key2022031319461130100_b10","doi-asserted-by":"crossref","unstructured":"Gordon, M. and Pathak, P. (1999), \u201cFinding information on the world wide web: the retrieval effectiveness of search engines\u201d, Information Processing & Management, Vol. 35 No. 2, pp. 141\u201080.","DOI":"10.1016\/S0306-4573(98)00041-7"},{"key":"key2022031319461130100_b11","doi-asserted-by":"crossref","unstructured":"Granka, L.A., Joachims, T. and Gay, G. (2004), \u201cEye\u2010tracking analysis of user behavior in WWW search\u201d, Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM Press, New York, NY, pp. 478\u20109.","DOI":"10.1145\/1008992.1009079"},{"key":"key2022031319461130100_b12","unstructured":"Griesbaum, J. (2004), \u201cEvaluation of three German search engines: Altavista.de, Google.de and Lycos.de\u201d, Information Research, Vol. 9 No. 4, available at: www.informationr.net\/ir\/9\u20104\/paper189.html."},{"key":"key2022031319461130100_b13","unstructured":"Griesbaum, J., Rittberger, M. and Bekavac, B. (2002), \u201cDeutsche Suchmaschinen im Vergleich: AltaVista.de, Fireball.de und Lycos.de\u201d, Procedings of the 8 Internationales Symposium f\u00fcr Informationswissenschaft, pp. 201\u201023."},{"key":"key2022031319461130100_b14","unstructured":"Guggenheim, E. and Bar\u2010Ilan, J. (2005), \u201cTauglichkeit von Suchmaschinen f\u00fcr deutschsprachige Anfragen\u201d, Information Wissenschaft und Praxis, Vol. 56 No. 1, pp. 35\u201040."},{"key":"key2022031319461130100_b15","doi-asserted-by":"crossref","unstructured":"Gulli, A. and Signorini, A. (2005), \u201cThe indexable web is more than 11.5 billion pages\u201d, Proceedings of the 14th International Conference on World Wide Web, Chiba, Japan, pp. 902\u20103.","DOI":"10.1145\/1062745.1062789"},{"key":"key2022031319461130100_b16","unstructured":"Hawking, D. and Craswell, N. (2005), \u201cThe very large collection and web tracks\u201d, in Voorhees, E.M. and Harman, D.K. (Eds), TREC: Experiment and Evaluation in Information Retrieval, MIT Press, Cambridge, MA, pp. 199\u2010231."},{"key":"key2022031319461130100_b17","doi-asserted-by":"crossref","unstructured":"Hawking, D., Craswell, N., Bailey, P. and Griffiths, K. (2001), \u201cMeasuring search engine quality\u201d, Information Retrieval, Vol. 4 No. 1, pp. 33\u201059.","DOI":"10.1023\/A:1011468107287"},{"key":"key2022031319461130100_b18","doi-asserted-by":"crossref","unstructured":"Jacs\u00f3, P. (2005), \u201cOptions for presenting search results\u201d, Online Information Review, Vol. 29 No. 3, pp. 311\u20109.","DOI":"10.1108\/14684520510607614"},{"key":"key2022031319461130100_b19","doi-asserted-by":"crossref","unstructured":"J\u00e4rvelin, K. and Kek\u00e4il\u00e4inen, J. (2002), \u201cCumulated gain\u2010based evaluation of IR techniques\u201d, ACM Transactions on Information Systems, Vol. 20 No. 4, pp. 422\u201046.","DOI":"10.1145\/582415.582418"},{"key":"key2022031319461130100_b20","unstructured":"Lazarinis, F. (2007), \u201cWeb retrieval systems and the Greek language: do they have an understanding?\u201d, Journal of Information Science, Vol. 30, pp. 1\u201015."},{"key":"key2022031319461130100_b21","doi-asserted-by":"crossref","unstructured":"Leighton, H.V. and Srivastava, J. (1999), \u201cFirst 20 precision among world wide web search services (search engines)\u201d, Journal of the American Society for Information Science, Vol. 50 No. 10, pp. 870\u201081.","DOI":"10.1002\/(SICI)1097-4571(1999)50:10<870::AID-ASI4>3.0.CO;2-G"},{"key":"key2022031319461130100_b22","doi-asserted-by":"crossref","unstructured":"Lewandowski, D. (2006), \u201cQuery types and search topics of German web search engine users\u201d, Information Services & Use, Vol. 26 No. 4, pp. 261\u20109.","DOI":"10.3233\/ISU-2006-26401"},{"key":"key2022031319461130100_b23","unstructured":"Lewandowski, D. (2007a), \u201cMit welchen Kennzahlen l\u00e4sst sich die Qualit\u00e4t von Suchmaschinen messen?\u201d, in Machill, M. and Beiler, M. (Eds), Die Macht der Suchmaschinen (The Power of Search Engines), von Halem, K\u00f6ln, pp. 243\u201058."},{"key":"key2022031319461130100_b24","unstructured":"Lewandowski, D. (2007b), \u201cTrefferpr\u00e4sentation in Web\u2010Suchmaschine\u201d, Proceedings of Information in Wissenschaft, Bildung und Wirtschaft, Online\u2010Tagung der DGI 2007, Vol. 29, DGI, Frankfurt am Main."},{"key":"key2022031319461130100_b25","doi-asserted-by":"crossref","unstructured":"Lewandowski, D. and H\u00f6chst\u00f6tter, N. (2007), \u201cWeb Searching: a quality measurement perspective\u201d, in Spink, A. and Zimmer, M. (Eds), Web Searching: Interdisciplinary Perspectives, Springer, Dordrecht.","DOI":"10.1007\/978-3-540-75829-7_16"},{"key":"key2022031319461130100_b26","doi-asserted-by":"crossref","unstructured":"Lewandowski, D., Wahlig, H. and Meyer\u2010Bautor, G. (2006), \u201cThe freshness of web search engine databases\u201d, Journal of Information Science, Vol. 32 No. 2, pp. 133\u201050.","DOI":"10.1177\/0165551506062326"},{"key":"key2022031319461130100_b27","unstructured":"Machill, M., Neuberger, C., Schweiger, W. and Wirth, W. (2003), \u201cWegweiser im Netz: qualit\u00e4t und Nutzung von Suchmaschinen\u201d, in Machill, M. and Welp, C. (Eds), Wegweiser im Netz, Bertelsmann Stiftung, G\u00fctersloh, pp. 17\u2010490."},{"key":"key2022031319461130100_b28","doi-asserted-by":"crossref","unstructured":"Mizzaro, S. (1997), \u201cRelevance: the whole history\u201d, Journal of the American Society for Information Science, Vol. 48 No. 9, pp. 810\u201032.","DOI":"10.1002\/(SICI)1097-4571(199709)48:9<810::AID-ASI6>3.0.CO;2-U"},{"key":"key2022031319461130100_b29","doi-asserted-by":"crossref","unstructured":"Rose, D.E. and Levinson, D. (2004), \u201cUnderstanding user goals in web search\u201d, Proceedings of the thirteenth International World Wide Web Conference, ACM Press, New York, NY, pp. 13\u201019.","DOI":"10.1145\/988672.988675"},{"key":"key2022031319461130100_b30","doi-asserted-by":"crossref","unstructured":"Schmidt\u2010Maenz, N. and Koch, M. (2006), \u201cA general classification of (search) queries and terms\u201d, Proceedings of the Third International Conference on Information Technologies: Next Generations, Las Vegas, NV, pp. 375\u201081.","DOI":"10.1109\/ITNG.2006.9"},{"key":"key2022031319461130100_b31","doi-asserted-by":"crossref","unstructured":"Spink, A., Wolfram, D., Jansen, M. and Saracevic, T. (2001), \u201cSearching the web: the public and their queries\u201d, Journal of the American Society for Information Science and Technology, Vol. 53 No. 2, pp. 226\u201034.","DOI":"10.1002\/1097-4571(2000)9999:9999<::AID-ASI1591>3.0.CO;2-R"},{"key":"key2022031319461130100_b32","unstructured":"Stock, M. and Stock, W.G. (2000), \u201cInternet\u2010Suchwerkzeuge im Vergleich, Teil 1: Retrievaltest mit Known Item Searches\u201d, Password, Vol. 15 No. 11, pp. 23\u201031."},{"key":"key2022031319461130100_b33","unstructured":"Sullivan, D. (2006), \u201cHitwise search engine ratings\u201d, Search Engine Watch, available at: http:\/\/searchenginewatch.com\/showPage.html?page\u2009=\u20093099931."},{"key":"key2022031319461130100_b34","unstructured":"Sullivan, D. (2007), \u201cMajor search engines and directories\u201d, Search Engine Watch, available at: http:\/\/searchenginewatch.com\/showPage.html?page\u2009=\u20092156221."},{"key":"key2022031319461130100_b35","doi-asserted-by":"crossref","unstructured":"Tague\u2010Sucliffe, J. (1992), \u201cThe pragmatics of information retrieval experimentation, revisited\u201d, Information Processing & Management, Vol. 28 No. 4, pp. 467\u201090.","DOI":"10.1016\/0306-4573(92)90005-K"},{"key":"key2022031319461130100_b36","doi-asserted-by":"crossref","unstructured":"Vaughan, L. (2004), \u201cNew measurements for search engine evaluation proposed and tested\u201d, Information Processing & Management, Vol. 40 No. 4, pp. 677\u201091.","DOI":"10.1016\/S0306-4573(03)00043-8"},{"key":"key2022031319461130100_b37","doi-asserted-by":"crossref","unstructured":"Vaughan, L. and Thelwall, M. (2004), \u201cSearch engine coverage bias: evidence and possible causes\u201d, Information Processing & Management, Vol. 40 No. 4, pp. 693\u2010707.","DOI":"10.1016\/S0306-4573(03)00063-3"},{"key":"key2022031319461130100_b38","unstructured":"V\u00e9ronis, J. (2006), \u201cA comparative study of six search engines\u201d, available at: www.up.univ\u2010mrs.fr\/veronis\/pdf\/2006\u2010comparative\u2010study.pdf."},{"key":"key2022031319461130100_b39","unstructured":"Webhits Web\u2010Barometer (2006), available at: www.webhits.de\/deutsch\/index.shtml?webstats.html."},{"key":"key2022031319461130100_b40","unstructured":"Wolff, C. (2000), \u201cVergleichende Evaluierung von Such\u2010 und Metasuchmaschinen\u201d, Proceedings of 7. Internationales Symposium f\u00fcr Informationswissenschaft, Darmstadt, Germany, Universit\u00e4tsverlag, Konstanz, pp. 31\u201048."}],"container-title":["Journal of Documentation"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/00220410810912451","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/00220410810912451\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/00220410810912451\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T23:38:00Z","timestamp":1753400280000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/jd\/article\/64\/6\/915-937\/204391"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,10,17]]},"references-count":40,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2008,10,17]]}},"alternative-id":["10.1108\/00220410810912451"],"URL":"https:\/\/doi.org\/10.1108\/00220410810912451","relation":{},"ISSN":["0022-0418"],"issn-type":[{"value":"0022-0418","type":"print"}],"subject":[],"published":{"date-parts":[[2008,10,17]]}}}