{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,6]],"date-time":"2025-11-06T15:51:20Z","timestamp":1762444280110,"version":"3.41.2"},"reference-count":23,"publisher":"Emerald","issue":"6","license":[{"start":{"date-parts":[[2011,11,29]],"date-time":"2011-11-29T00:00:00Z","timestamp":1322524800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2011,11,29]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-heading\">Purpose<\/jats:title><jats:p>This paper seeks to evaluate to what extent Google retrieved correct answers to queries inferred from factual and topical requests in a digital Ask\u2010a\u2010Librarian service.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Design\/methodology\/approach<\/jats:title><jats:p>In total, 100 factual and 100 topical questions were picked from a digital reference service run by public libraries. The inferred queries simulated average web queries. They were expressed as separate keywords and as questions. The top ten retrieval results were observed for each answer. The inspection was stopped when the first correct answer was identified.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Findings<\/jats:title><jats:p>Google retrieved correct answers to 42 percent of the topical questions and 29 percent of factual questions by keyword queries. The performance of queries in question form was considerably weaker. The results concerning the characteristics of queries and retrieval effectiveness are also presented. Evaluations indicate that the public library reference services answered at least 55 percent of the questions correctly. Thus Google did not outperform the Ask\u2010a\u2010Librarian service.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Originality\/value<\/jats:title><jats:p>The study introduces a new way of evaluating search engines by comparing their performance with other related services such as an Ask\u2010a\u2010Librarian service.<\/jats:p><\/jats:sec>","DOI":"10.1108\/14684521111193201","type":"journal-article","created":{"date-parts":[[2011,11,26]],"date-time":"2011-11-26T07:09:44Z","timestamp":1322291384000},"page":"928-941","source":"Crossref","is-referenced-by-count":9,"title":["Comparing Google to a digital reference service for answering factual and topical requests by keyword and question queries"],"prefix":"10.1108","volume":"35","author":[{"given":"Pertti","family":"Vakkari","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","reference":[{"key":"key2022021920220943200_b1","doi-asserted-by":"crossref","unstructured":"Arnold, J. and Kaske, N. (2005), \u201cEvaluating the quality of a chat service\u201d, Portal, Vol. 5 No. 2, pp. 177\u201093.","DOI":"10.1353\/pla.2005.0017"},{"key":"key2022021920220943200_b2","unstructured":"Belkin, N.J., Cool, C., Jeng, J., Keller, D., Kelly, D., Kim, J., Lee, H.\u2010J., Tang, M.\u2010C. and Yuan, X.\u2010J. (2002), \u201cRutger's TREC 2001 interactive track experience\u201d, in Voorhees, E.M. and Harman, D.M. (Eds), The Tenth Text Retrieval Conference, TREC 2001, GPO, Washington DC, pp. 465\u201072."},{"key":"key2022021920220943200_b3","doi-asserted-by":"crossref","unstructured":"Bendersky, M. and Croft, B. (2009), \u201cAnalysis of long queries in a large scale search log\u201d, Proceedings of the 2009 Workshop on Web Search Click Data, ACM, New York, NY, pp. 8\u201014.","DOI":"10.1145\/1507509.1507511"},{"key":"key2022021920220943200_b4","doi-asserted-by":"crossref","unstructured":"Broder, A. (2002), \u201cA taxonomy of web search\u201d, SIGIR Forum, Vol. 36 No. 2, pp. 3\u201010.","DOI":"10.1145\/792550.792552"},{"key":"key2022021920220943200_b5","unstructured":"Chu, H. and Rosenthal, M. (1996), \u201cSearch engines for the world wide web: a comparative study and evaluation methodology\u201d, ASIS 1996 Annual Conference Proceedings, available at: www.asis.org\/annual\u201096\/ElectronicProceedings\/chu.html (accessed 9 August 2011)."},{"key":"key2022021920220943200_b6","doi-asserted-by":"crossref","unstructured":"D'Elia, G., J\u00f6rgensen, C. and Woelfel, J. (2002), \u201cThe impact of the internet on public library use\u201d, Journal of the American Society for Information Science and Technology, Vol. 53 No. 10, pp. 802\u201020.","DOI":"10.1002\/asi.10102"},{"key":"key2022021920220943200_b7","doi-asserted-by":"crossref","unstructured":"Gordon, M. and Pathak, P. (1999), \u201cFinding information on the world wide web: the retrieval effectiveness of search engines\u201d, Information Processing & Management, Vol. 35 No. 2, pp. 141\u201080.","DOI":"10.1016\/S0306-4573(98)00041-7"},{"key":"key2022021920220943200_b8","doi-asserted-by":"crossref","unstructured":"Harper, F.M., Raban, D., Rafaeli, S. and Konstan, J. (2008), \u201cPredictors of answer quality in online Q&A sites\u201d, Proceeding of the 26th SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, pp. 865\u201074.","DOI":"10.1145\/1357054.1357191"},{"key":"key2022021920220943200_b9","doi-asserted-by":"crossref","unstructured":"Janes, J., Hill, C. and Rolfe, A. (2001), \u201cAsk\u2010an\u2010Expert services analysis\u201d, Journal of the American Society for Information Science and Technology, Vol. 52 No. 13, pp. 1106\u201021.","DOI":"10.1002\/asi.1177"},{"key":"key2022021920220943200_b10","unstructured":"Kaske, N. and Arnold, J. (2002), \u201cAn unobtrusive evaluation of online real time library reference services\u201d, paper presented at the Library Research Round Table, ALA Annual Conference, Atlanta, GA, available at: www.lib.umd.edu\/groups\/digref\/LRRT.html (accessed 10 April 2009)."},{"key":"key2022021920220943200_b11","unstructured":"Katz, W.A. (2002), Introduction to Reference Work. Vol. 1: Basic Information Services, McGraw\u2010Hill, Boston, MA."},{"key":"key2022021920220943200_b12","doi-asserted-by":"crossref","unstructured":"Kellar, M., Watters, K. and Shepherd, M. (2007), \u201cA field study characterizing web\u2010based information seeking tasks\u201d, Journal of the American Society for Information Science and Technology, Vol. 58 No. 7, pp. 999\u20101018.","DOI":"10.1002\/asi.20590"},{"key":"key2022021920220943200_b13","unstructured":"Krippendorf, K. (2004), Content Analysis, Sage, Thousand Oaks, CA."},{"key":"key2022021920220943200_b14","doi-asserted-by":"crossref","unstructured":"Kwon, N. (2007), \u201cPublic library patrons' use of collaborative chat reference service: the effectiveness of question answering by question type\u201d, Library & Information Science Research, Vol. 29 No. 1, pp. 70\u201091.","DOI":"10.1016\/j.lisr.2006.08.012"},{"key":"key2022021920220943200_b15","unstructured":"Lancaster, W. and Warner, A. (1993), Information Retrieval Today, Information Resources Press, Arlington, VA."},{"key":"key2022021920220943200_b16","doi-asserted-by":"crossref","unstructured":"Leighton, W. and Srivastava, J. (1999), \u201cFirst 20 precision among world wide web search services\u201d, Journal of the American Society for Information Science, Vol. 50 No. 10, pp. 870\u201081.","DOI":"10.1002\/(SICI)1097-4571(1999)50:10<870::AID-ASI4>3.0.CO;2-G"},{"key":"key2022021920220943200_b17","doi-asserted-by":"crossref","unstructured":"Lewandowski, D. (2008), \u201cThe retrieval effectiveness of web search engines: considering results descriptions\u201d, Journal of Documentation, Vol. 64 No. 6, pp. 915\u201037.","DOI":"10.1108\/00220410810912451"},{"key":"key2022021920220943200_b18","doi-asserted-by":"crossref","unstructured":"Lewandowski, D. and H\u00f6chst\u00f6tter, N. (2008), \u201cWeb searching: a quality measurement perspective\u201d, Web Search. Information Science and Knowledge Management, Vol. 14, IV, Springer, Berlin, pp. 309\u201040.","DOI":"10.1007\/978-3-540-75829-7_16"},{"key":"key2022021920220943200_b19","doi-asserted-by":"crossref","unstructured":"Rose, D. and Levinson, D. (2004), \u201cUnderstanding user goals in web search\u201d, WWW 2004, ACM, New York, NY, pp. 13\u201019.","DOI":"10.1145\/988672.988675"},{"key":"key2022021920220943200_b20","doi-asserted-by":"crossref","unstructured":"Shachaf, P. (2009), \u201cThe paradox of expertise: is the Wikipedia Reference Desk as good as your library?\u201d, Journal of Documentation, Vol. 65 No. 6, pp. 977\u201096.","DOI":"10.1108\/00220410910998951"},{"key":"key2022021920220943200_b21","unstructured":"Sormunen, E. (2000), \u201cA method of measuring wide range performance of Boolean queries in full\u2010text databases\u201d, Acta Universitatis Tamperensis 748, PhD thesis, Tampere University Press, Tampere."},{"key":"key2022021920220943200_b22","doi-asserted-by":"crossref","unstructured":"Spink, A. and Jansen, B. (2004), Web Search: Public Searching of the Web, Kluwer, Dordrecht.","DOI":"10.1007\/1-4020-2269-7"},{"key":"key2022021920220943200_b23","doi-asserted-by":"crossref","unstructured":"Vakkari, P. and Taneli, M. (2009), \u201cComparing Google to Ask\u2010a\u2010Librarian service for answering factual and topical questions\u201d, in Agosti, M., Bobinha, J., Kapidakis, S., Pappatheodorou, C. and Tsakonas, G. (Eds), Proceedings of ECDL 2009, Lecture Notes in Computer Science 5714, Springer, Berlin, pp. 352\u201063.","DOI":"10.1007\/978-3-642-04346-8_34"}],"container-title":["Online Information Review"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/14684521111193201","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/14684521111193201\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/14684521111193201\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T00:41:47Z","timestamp":1753404107000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/oir\/article\/35\/6\/928-941\/314099"}},"subtitle":[],"editor":[{"given":"Dirk","family":"Lewandowski","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2011,11,29]]},"references-count":23,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2011,11,29]]}},"alternative-id":["10.1108\/14684521111193201"],"URL":"https:\/\/doi.org\/10.1108\/14684521111193201","relation":{},"ISSN":["1468-4527"],"issn-type":[{"type":"print","value":"1468-4527"}],"subject":[],"published":{"date-parts":[[2011,11,29]]}}}