{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T04:53:08Z","timestamp":1773809588195,"version":"3.50.1"},"reference-count":40,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2017,10,24]],"date-time":"2017-10-24T00:00:00Z","timestamp":1508803200000},"content-version":"vor","delay-in-days":296,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["asistdl.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Proc. Assoc. Info. Sci. Tech."],"published-print":{"date-parts":[[2017,1]]},"abstract":"<jats:title>ABSTRACT<\/jats:title><jats:p>The objective of this research was to find out how the two search engines, Google and Bing, perform when users work freely on pre\u2010defined tasks, and judge the relevance of the results immediately after finishing their search session. In a user study, 64 participants conducted two search tasks each, and then judged the results on the following: (1) the quality of the results they selected in their search sessions, (2) the quality of the results they were presented with in their search sessions (but which they did not click on), (3) the quality of the results from the competing search engine for their queries (which they did not see in their search session). We found that users heavily relied on Google, that Google produced more relevant results than Bing, that users were well able to select relevant results from the results lists and that users judged the relevance of results lower when they regarded a task as difficult and did not find the correct information.<\/jats:p>","DOI":"10.1002\/pra2.2017.14505401044","type":"journal-article","created":{"date-parts":[[2017,10,24]],"date-time":"2017-10-24T03:35:36Z","timestamp":1508816136000},"page":"405-414","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Does it matter which search engine is used? A user study using post\u2010task relevance judgments"],"prefix":"10.1002","volume":"54","author":[{"given":"Sebastian","family":"S\u00fcnkler","sequence":"first","affiliation":[{"name":"Hamburg University of Applied Sciences  Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dirk","family":"Lewandowski","sequence":"additional","affiliation":[{"name":"Hamburg University of Applied Sciences  Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2017,10,24]]},"reference":[{"key":"e_1_2_9_2_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.20608"},{"key":"e_1_2_9_3_1","doi-asserted-by":"publisher","DOI":"10.1108\/JD-08\u20132016\u20130099"},{"key":"e_1_2_9_4_1","unstructured":"Belkin N. Cole M. &Liu J.(2009).A model for evaluation of interactive information retrieval. InProceedings of the SIGIR 2009 workshop on the future of IR evaluation(pp.7\u20138)."},{"issue":"3","key":"e_1_2_9_5_1","article-title":"The IIR evaluation model: A framework for evaluation of interactive information retrieval systems","volume":"8","author":"Borlund P.","year":"2003","journal-title":"Information Research"},{"key":"e_1_2_9_6_1","doi-asserted-by":"crossref","unstructured":"Capra R.(2010). HCI browser: A tool for studying web search behavior. InASIST 2010(pp.1\u20132).","DOI":"10.1002\/meet.14504701444"},{"key":"e_1_2_9_7_1","unstructured":"comScore. (2013).Europe digital future in focus: Key insights from 2012 and what they mean for the coming year. Report. Retrieved fromhttp:\/\/www.comscore.com\/layout\/set\/popup\/Request\/Presentations\/2013\/2013_Europe_Digital_Future_in_Focus_Report_PDF_Request?req=slides&pre=2013+Europe+Digital+Future+in+Focus"},{"key":"e_1_2_9_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1059981.1059982"},{"key":"e_1_2_9_9_1","doi-asserted-by":"crossref","unstructured":"Goel S. Broder A. Gabrilovich E. &Pang B.(2010).Anatomy of the long tail: Ordinary people with extraordinary tastes. InProceedings of the third ACM international conference on web search and data mining(pp.201\u2013210).ACM.","DOI":"10.1145\/1718487.1718513"},{"key":"e_1_2_9_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(98)00041-7"},{"issue":"4","key":"e_1_2_9_11_1","volume":"9","author":"Griesbaum J.","year":"2004","journal-title":"Information Research"},{"key":"e_1_2_9_12_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1011468107287"},{"key":"e_1_2_9_13_1","unstructured":"Jansen B. J.(2006). The wrapper: An open source application for logging user\u2013system interactions during searching studies. InLogging traces of web activity: The mechanics of data collection workshop at WWW 2006(pp.1\u20134)."},{"key":"e_1_2_9_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1232722.1232725"},{"key":"e_1_2_9_15_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.21081"},{"key":"e_1_2_9_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1229179.1229181"},{"key":"e_1_2_9_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1314215.1314224"},{"key":"e_1_2_9_18_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000012"},{"key":"e_1_2_9_19_1","doi-asserted-by":"publisher","DOI":"10.1108\/00220410810912451"},{"key":"e_1_2_9_20_1","doi-asserted-by":"crossref","unstructured":"Lewandowski D.(2011).The influence of commercial intent of search results on their perceived relevance. InProceedings of the 2011 iConference(pp.452\u2013458).ACM.http:\/\/doi.org\/10.1145\/1940761.1940823","DOI":"10.1145\/1940761.1940823"},{"key":"e_1_2_9_21_1","doi-asserted-by":"publisher","DOI":"10.4018\/978-1-4666-0330-1.ch020"},{"key":"e_1_2_9_22_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23304"},{"key":"e_1_2_9_23_1","doi-asserted-by":"publisher","DOI":"10.3233\/ISU-130691"},{"key":"e_1_2_9_24_1","doi-asserted-by":"publisher","DOI":"10.1177\/0267323104045258"},{"key":"e_1_2_9_25_1","unstructured":"Mayr P. &Tosques F.(2005).Google web APIs \u2010 An instrument for webometric analyses?InP.Ingwersen&B.Larsen(Eds.) Proceedings of the 10th international conference of the International Society for Scientometrics and Informetrics (ISSI 2005). Stockholm (Sweden) (pp.677\u2013678)."},{"key":"e_1_2_9_26_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1083-6101.2007.00351.x"},{"key":"e_1_2_9_27_1","unstructured":"Petrescu P.(2014).Google organic click\u2010through rates in 2014 \u2010Moz. Retrieved April 25 2016 fromhttps:\/\/moz.com\/blog\/google-organic-click-through-rates-in-2014"},{"key":"e_1_2_9_28_1","unstructured":"Purcell K. Brenner J. &Raine L.(2012).Search engine use 2012. Search. Washington DC. Retrieved fromhttp:\/\/pewinternet.org\/\u223c\/media\/Files\/Reports\/2012\/PIP_Search_Engine_Use_2012.pdf"},{"key":"e_1_2_9_29_1","first-page":"227","volume-title":"CLEF 2016","author":"Schaer P.","year":"2016"},{"key":"e_1_2_9_30_1","doi-asserted-by":"crossref","unstructured":"Singer G. Danilov D. &Norbisrath U.(2012).Complex search: Aggregation discovery and synthesis.Proceedings of the Estonian Academy of Sciences 61(2) 89\u2013106.http:\/\/doi.org\/10.3176\/proc.2012.2.02","DOI":"10.3176\/proc.2012.2.02"},{"key":"e_1_2_9_31_1","doi-asserted-by":"crossref","unstructured":"Singer G. Norbisrath U. &Lewandowski D.(2012).Ordinary search engine users assessing difficulty effort and outcome for simple and complex search tasks. InProceedings of the fourth information interaction in context symposium(pp.110\u2013119).New York:ACM.","DOI":"10.1145\/2362724.2362746"},{"key":"e_1_2_9_32_1","doi-asserted-by":"publisher","DOI":"10.1177\/0165551512466974"},{"key":"e_1_2_9_33_1","doi-asserted-by":"crossref","unstructured":"Singer G. Norbisrath U. Vainikko E. Kikkas H. &Lewandowski D.(2011). Search Logger: Analysing exploratory search tasks. InSAC2011. Proceedings of the 2011 ACM symposium on applied computing(pp.751\u2013756).New York:ACM.","DOI":"10.1145\/1982185.1982350"},{"key":"e_1_2_9_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2005.11.001"},{"key":"e_1_2_9_35_1","first-page":"223","volume-title":"Handbuch Internet\u2010Suchmaschinen 3: Suchmaschinen zwischen Technik und Gesellschaft","author":"S\u00fcnkler S.","year":"2013"},{"key":"e_1_2_9_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(92)90005-K"},{"key":"e_1_2_9_37_1","unstructured":"Tawileh W. Griesbaum J. &Mandl T.(2010).Evaluation of five web search engines in Arabic language. InM.Atzm\u00fcller D.Benz A.Hotho &G.Stumme(Eds.) Proceedings of LWA2010(pp.1\u20138).Kassel Germany. Retrieved fromhttp:\/\/www.kde.cs.uni-kassel.de\/conf\/lwa10\/papers\/ir1.pdf"},{"key":"e_1_2_9_38_1","volume-title":"Suchen, finden\u2013glauben?: Die Rolle der Glaubwurdigkeit von Suchergebnissen bei der Nutzung von Suchmaschinen","author":"Tremel A.","year":"2010"},{"key":"e_1_2_9_39_1","doi-asserted-by":"crossref","unstructured":"Voorhees E. M.(2009).I come not to bury Cranfield but to praise it. InProceedings of the workshop on human\u2010computer interaction and retrieval(pp.13\u201316).http:\/\/doi.org\/10.1080\/2158379X.2013.849369","DOI":"10.1080\/2158379X.2013.849369"},{"key":"e_1_2_9_40_1","doi-asserted-by":"publisher","DOI":"10.1111\/jcc4.12006"},{"key":"e_1_2_9_41_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781139525305"}],"container-title":["Proceedings of the Association for Information Science and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fpra2.2017.14505401044","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/pra2.2017.14505401044","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/pra2.2017.14505401044","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/pra2.2017.14505401044","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T16:27:33Z","timestamp":1761064053000},"score":1,"resource":{"primary":{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/10.1002\/pra2.2017.14505401044"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,1]]},"references-count":40,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2017,1]]}},"alternative-id":["10.1002\/pra2.2017.14505401044"],"URL":"https:\/\/doi.org\/10.1002\/pra2.2017.14505401044","archive":["Portico"],"relation":{},"ISSN":["2373-9231","2373-9231"],"issn-type":[{"value":"2373-9231","type":"print"},{"value":"2373-9231","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,1]]},"assertion":[{"value":"2017-10-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}