{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T20:47:46Z","timestamp":1779914866598,"version":"3.53.1"},"reference-count":53,"publisher":"Emerald","issue":"2","license":[{"start":{"date-parts":[[2000,4,1]],"date-time":"2000-04-01T00:00:00Z","timestamp":954547200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2000,4,1]]},"abstract":"<jats:p>The literature of the evaluation of Internet search engines is reviewed. Although there have been many studies, there has been little consistency in the way such studies have been carried out. This problem is exacerbated by the fact that recall is virtually impossible to calculate in the fast changing Internet environment, and therefore the traditional Cranfield type of evaluation is not usually possible. A variety of alternative evaluation methods has been suggested to overcome this difficulty. The authors recommend that a standardised set of tools is developed for the evaluation of web search engines so that, in future, comparisons can be made between search engines more effectively, and that variations in performance of any given search engine over time can be tracked. The paper itself does not provide such a standard set of tools, but it investigates the issues and makes preliminary recommendations of the types of tools needed.<\/jats:p>","DOI":"10.1108\/00220410010803810","type":"journal-article","created":{"date-parts":[[2007,6,19]],"date-time":"2007-06-19T11:38:37Z","timestamp":1182253117000},"page":"190-211","source":"Crossref","is-referenced-by-count":52,"title":["The evaluation of WWW search engines"],"prefix":"10.1108","volume":"56","author":[{"given":"C.","family":"Oppenheim","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"A.","family":"Morris","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"C.","family":"McKnight","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"S.","family":"Lowley","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"140","reference":[{"issue":"4","key":"p_1","first-page":"373","volume":"39","author":"Teutenberg F","year":"1997","journal-title":"Wirtschaftsinformatik"},{"key":"p_2","volume-title":"An investigation into World Wide Web search engine use from within the UK JISC-funded project undertaken by UKERNA","author":"Stobart S.","year":"1996"},{"key":"p_3","first-page":"4","volume":"104","author":"Furner J","year":"1997","journal-title":"VINE"},{"key":"p_4","first-page":"65","author":"Gudivada V.N.","year":"1997","journal-title":"IEEE Internet Computing"},{"key":"p_5","doi-asserted-by":"publisher","DOI":"10.1108\/eb040568"},{"key":"p_6","doi-asserted-by":"publisher","DOI":"10.1016\/S0169-7552(98)00096-8"},{"key":"p_7","doi-asserted-by":"publisher","DOI":"10.1016\/S0950-5849(97)00061-X"},{"key":"p_8","volume-title":"http:\/\/www.searchenginewatch.com\/ (visited","author":"Search Engine Watch","year":"1999"},{"key":"p_10","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(1998)49:11<973::AID-ASI3>3.0.CO;2-Z"},{"key":"p_11","volume-title":"Excite, HotBot, Infoseek, Lycos.","author":"Leighton H.V.","year":"1997"},{"key":"p_12","first-page":"3","volume":"32","author":"Harter S.P.","year":"1997","journal-title":"Annual Review of Information Science and Technology"},{"key":"p_13","volume-title":"Progress and problems in information retrieval","author":"Ellis D.","year":"1996"},{"key":"p_14","first-page":"127","volume-title":"ASIS 1996 Annual Conference Proceedings","author":"Chu H.","year":"1996"},{"key":"p_15","doi-asserted-by":"publisher","DOI":"10.1177\/016555159802400604"},{"issue":"1","key":"p_16","first-page":"42","volume":"21","author":"Lauren J.V","year":"1998","journal-title":"Database"},{"key":"p_17","doi-asserted-by":"publisher","DOI":"10.1300\/J136v02n01_02"},{"key":"p_18","doi-asserted-by":"publisher","DOI":"10.1007\/BF02458356"},{"key":"p_19","doi-asserted-by":"publisher","DOI":"10.1108\/eb051463"},{"key":"p_20","first-page":"136","volume-title":"ASIS 1996 Annual Conference Proceedings","author":"Ding W.","year":"1996"},{"key":"p_21","volume-title":"Webnet 96 Conference","author":"Gauch S.","year":"1996"},{"issue":"6","key":"p_22","first-page":"58","volume":"16","author":"Tomaiuolo N.G.","year":"1996","journal-title":"Computers in Libraries"},{"key":"p_23","volume-title":"Search engine comparison: testing retrieval and accuracy","author":"Westera G.","year":"1996"},{"key":"p_24","volume-title":"July","author":"Nasios Y.","year":"1998"},{"key":"p_26","volume-title":"Signal detection and recognition by human observers","author":"Swets J.A.","year":"1964"},{"key":"p_28","first-page":"224","volume-title":"The digital revolution: proceedings of the ASIS Mid-Year Meeting, San Diego, CA","author":"Meghaghab D.B.","year":"1996"},{"key":"p_30","volume-title":"A technique for measuring the relative size and overlap of public Web search engines","author":"Bharat K.","year":"1997"},{"key":"p_31","doi-asserted-by":"publisher","DOI":"10.1016\/S0169-7552(98)00127-5"},{"key":"p_32","doi-asserted-by":"publisher","DOI":"10.1109\/35.739314"},{"key":"p_33","doi-asserted-by":"publisher","DOI":"10.1126\/science.280.5360.98"},{"issue":"6","key":"p_34","first-page":"29","volume":"6","author":"Feldman S","year":"1998","journal-title":"Searcher"},{"issue":"1","key":"p_35","first-page":"40","volume":"19","author":"Kimmel S","year":"1996","journal-title":"Database"},{"key":"p_36","first-page":"173","volume":"17","author":"Tunender H.","year":"1998","journal-title":"Information Technology and Libraries"},{"key":"p_37","doi-asserted-by":"publisher","DOI":"10.1002\/asi.5090190108"},{"key":"p_38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.46895\/lis.37.1","volume":"37","author":"Agata T.","year":"1997","journal-title":"Library and Information Science"},{"key":"p_39","volume-title":"Evaluation of search engines for finding medical information on the Web","author":"Humphries K.A.","year":"1997"},{"key":"p_40","doi-asserted-by":"publisher","DOI":"10.1300\/J136v03n02_09"},{"key":"p_41","first-page":"13","volume-title":"Proceedings of the 61st Annual Meeting of the American Society for Information Science, 35","author":"MacCall S.L.","year":"1998"},{"key":"p_42","doi-asserted-by":"publisher","DOI":"10.1108\/EUM0000000007144"},{"key":"p_44","volume-title":"Measuring the Web","author":"Bharat K.","year":"1998"},{"key":"p_45","volume-title":"Best search engines for finding scientific information on the Web","author":"Lebedev A.","year":"1997"},{"key":"p_46","volume-title":"http:\/\/www.searchenginewatch.com\/sereport\/9805-size.html (visited","author":"Search","year":"1999"},{"key":"p_47","volume-title":"http:\/\/www.albany. edu\/library\/internet\/choose.html (visited","author":"How","year":"1999"},{"key":"p_48","doi-asserted-by":"publisher","DOI":"10.1108\/eb024608"},{"key":"p_49","doi-asserted-by":"publisher","DOI":"10.1108\/eb040633"},{"key":"p_51","doi-asserted-by":"publisher","DOI":"10.1108\/EUM0000000006993"},{"key":"p_52","unstructured":"Golovchinsky, G. From information retrieval to hypertext and back again: the role of interaction in the information exploration interface. PhD Thesis,University of Toronto, 1996. http:\/\/anarch.ie.utoronto.ca\/people\/golovch\/ thesis\/final\/ (visited 30 July 1999)."},{"key":"p_53","first-page":"348","volume-title":"Proceedings of the ASIS Annual Meeting, 35","author":"Su L.T.","year":"1998"},{"key":"p_54","first-page":"205","volume-title":"Online Information '97 Proceedings: 21st International Online Information Meeting. Oxford: Learned Information Europe Ltd","author":"Keily L.","year":"1997"},{"key":"p_55","doi-asserted-by":"publisher","DOI":"10.1108\/eb051467"},{"key":"p_56","volume-title":"The psychology of thinking","author":"Tikhomirov O.K.","year":"1988"},{"key":"p_57","volume-title":"http:\/\/www.unn.ac.uk\/features.htm (visited","author":"Web","year":"1999"},{"key":"p_58","doi-asserted-by":"publisher","DOI":"10.1108\/eb024446"},{"key":"p_59","doi-asserted-by":"publisher","DOI":"10.1108\/eb024466"}],"container-title":["Journal of Documentation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/00220410010803810\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/00220410010803810\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T23:37:05Z","timestamp":1753400225000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/jd\/article\/56\/2\/190-211\/226421"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2000,4,1]]},"references-count":53,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2000,4,1]]}},"alternative-id":["10.1108\/00220410010803810"],"URL":"https:\/\/doi.org\/10.1108\/00220410010803810","relation":{},"ISSN":["0022-0418"],"issn-type":[{"value":"0022-0418","type":"print"}],"subject":[],"published":{"date-parts":[[2000,4,1]]}}}