{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,2]],"date-time":"2025-08-02T19:02:46Z","timestamp":1754161366625,"version":"3.41.2"},"reference-count":32,"publisher":"Emerald","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2015,11,2]]},"abstract":"<jats:sec>\n                  <jats:title>Purpose<\/jats:title>\n                  <jats:p>\u2013 This study aims to explore a framework for evaluating and comparing two federated search tools (FSTs) using two different retrieval protocols: XML gateways and Z39.50. FSTs are meta-information retrieval systems developed to facilitate the searching of multiple resources through a single search box. FSTs allow searching of heterogeneous platforms, such as bibliographic and full-text databases, online public access catalogues, web search engines and open-access resources.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Design\/methodology\/approach<\/jats:title>\n                  <jats:p>\u2013 The proposed framework consists of three phases: the usability testing, retrievability performance assessment and overall comparison. The think-aloud protocol was implemented for usability testing and FSTs retrieval consistency, and precision tests were carried out to assess the retrievability performance for 20 real user queries.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Findings<\/jats:title>\n                  <jats:p>\u2013 Participants were directed to assign weights for the interface usability and system retrievability importance as indicators for FST evaluation. Results indicated that FSTs retrievability performance was of more importance than the interface usability. Participants assigned an average weight of 62 per cent for the system retrievability and 38 per cent for interface usability. In terms of the usability test, there was no significant difference between the two FSTs, while minor differences were found regarding retrieval consistency and precision at 11-point cut-off recall. The overall evaluation showed that the FST based on the XML gateway rated slightly higher than the FST based on the Z39.50 protocol.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Research limitations\/implications<\/jats:title>\n                  <jats:p>\u2013 This empirical study faced several limitations. First, the lack of participants\u2019 familiarity with usability testing created the need for a deep awareness and rigorous supervision. Second, the difficulties of empirically assessing participants\u2019 perspectives and future attitudes called for mixing between a formal task and the think-aloud protocol for participants in a real environment. This has been a challenge that faced the collection of the usability data including user behaviour, expectations and other empirical data. Third, the differences between the two FSTs in terms of number of connectors and advanced search techniques required setting rigorous procedures for testing FSTs retrieval consistency and precision.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Practical implications<\/jats:title>\n                  <jats:p>\u2013 This paper has practical implications in two dimensions. First, its results could be utilized by FST developers to enhance their product\u2019s performance. Second, the framework could be used by librarians to evaluate FSTs performance and capabilities. The framework enables them to compare between library systems in general and FSTs in particular. In addition to these practical implications, the authors encourage researchers to use and enhance the proposed framework.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Social implications<\/jats:title>\n                  <jats:p>\u2013 Librarians can use the proposed framework to empirically select an FST, involving users in the selection procedures of these information retrieval systems, so that it accords with users\u2019 perspectives and attitudes and serves the community better.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Originality\/value<\/jats:title>\n                  <jats:p>\u2013 The proposed framework could be considered a benchmark for FST evaluation.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1108\/el-12-2013-0211","type":"journal-article","created":{"date-parts":[[2015,12,9]],"date-time":"2015-12-09T02:59:13Z","timestamp":1449629953000},"page":"1079-1099","source":"Crossref","is-referenced-by-count":3,"title":["Evaluating federated search tools: usability and retrievability framework"],"prefix":"10.1108","volume":"33","author":[{"given":"Khaled A.","family":"Mohamed","sequence":"first","affiliation":[{"name":"Department of Library and Information Science, Fayoum University, Fayoum, Egypt"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ahmed","family":"Hassan","sequence":"additional","affiliation":[{"name":"School of Engineering, Ain Shams University, Cairo, Egypt"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","reference":[{"key":"2025072816225140800_b1","unstructured":"Abercrombie, S.E.\n           (2008), \u201cEvaluation of federated searching options for the school library\u201d,                   School Library Media Research                , Vol. 11, available at: www.ala.org\/aasl\/sites\/ala.org.aasl\/files\/content\/aaslpubsandjournals\/slr\/vol11\/SLMR_EvaluationFederated_V11.pdf (accessed 15 January 2013)."},{"key":"2025072816225140800_b2","doi-asserted-by":"crossref","unstructured":"Arguello, J.\n          , Capra, R. and Wu, W.C. (2013), \u201cFactors affecting aggregated search coherence and search behaviour\u201d, Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, pp. 1989-1998.","DOI":"10.1145\/2505515.2505693"},{"issue":"3","key":"2025072816225140800_b3","first-page":"347","article-title":"The FedLemur project: federated search in the real world","volume":"57","author":"","year":"2005","journal-title":"Journal of the American Society for Information Science and Technology"},{"key":"2025072816225140800_b4","doi-asserted-by":"crossref","unstructured":"Bouchon-Meunier, B.\n           (1998), \u201cAggregation and fusion of imperfect information\u201d,                   Studies in Fuzziness and Soft Computing                , Physica-Verlag Heidelberg, New York, NY.","DOI":"10.1007\/978-3-7908-1889-5"},{"issue":"4","key":"2025072816225140800_b5","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1080\/00987913.2006.10765074","article-title":"The one-box challenge: providing a federated search that benefits the research process","volume":"32","author":"","year":"2006","journal-title":"Serials Review"},{"key":"2025072816225140800_b6","doi-asserted-by":"crossref","unstructured":"Bron, M.\n          , Van Gorp, J., Nack, F., Baltussen, L.B. and de Rijke, M. (2013), \u201cAggregated search interface preferences in multi-session search tasks\u201d, Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 123-132.","DOI":"10.1145\/2484028.2484050"},{"issue":"3","key":"2025072816225140800_b7","doi-asserted-by":"crossref","first-page":"235","DOI":"10.5860\/rusq.51n3.235","article-title":"Beyond the search box","volume":"51","author":"","year":"2012","journal-title":"Reference & User Services Quarterly"},{"issue":"1","key":"2025072816225140800_b8","first-page":"31","article-title":"The future role of librarians in the virtual library environment","volume":"51","author":"","year":"2001","journal-title":"Australian Library Journal"},{"issue":"9","key":"2025072816225140800_b9","first-page":"10","article-title":"What we\u2019ve learned from doing usability testing on OpenURL resolvers and federated search engines","volume":"25","author":"","year":"2005","journal-title":"Computers in Libraries"},{"issue":"4","key":"2025072816225140800_b10","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1108\/14684520610686300","article-title":"MetaLib, WebFeat, and Google \u2013 the strengths and weaknesses of federated search engines compared with Google","volume":"30","author":"","year":"2006","journal-title":"Online Information Review"},{"key":"2025072816225140800_b11","doi-asserted-by":"crossref","unstructured":"Dwork, C.\n          , Kumar, R., Naor, M. and Sivakumar, D. (2001), \u201cRank aggregation methods for the web\u201d, Proceedings of the 10th International Conference on the World Wide Web,                   ACM                , pp. 613-622.","DOI":"10.1145\/371920.372165"},{"issue":"2","key":"2025072816225140800_b12","first-page":"16","article-title":"Federated search engines: federated searching aggregates multiple channels of information into a single searchable point","volume":"28","author":"","year":"2004","journal-title":"Online"},{"issue":"2","key":"2025072816225140800_b13","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1353\/pla.2013.0011","article-title":"Google vs. the library: student preferences and perceptions when doing research using Google and a federated search tool","volume":"13","author":"","year":"2013","journal-title":"Portal-Libraries and the Academy"},{"issue":"1","key":"2025072816225140800_b14","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1108\/02640470810851707","article-title":"Lessons learned: usability testing a federated search product","volume":"26","author":"","year":"2008","journal-title":"The Electronic Library"},{"issue":"6","key":"2025072816225140800_b15","first-page":"454","article-title":"Savvy searching","volume":"28","author":"","year":"2003","journal-title":"Online Information Review"},{"key":"2025072816225140800_b16","unstructured":"Jaffe, H.\n           and Mukherjee, R. (2013), \u201cSystem and method for dynamic context-sensitive federated search of multiple information repositories\u201d,                   US Patent Application                \u200810\/743, 196, available at: www.google.com\/patents\/US20050149496 (accessed 12 June 2013)."},{"issue":"3","key":"2025072816225140800_b17","first-page":"41","article-title":"Aggregated search: a new information retrieval paradigm","volume":"46","author":"","year":"2014","journal-title":"ACM Computing Surveys (CSUR"},{"key":"2025072816225140800_b18","unstructured":"Kumar, S.\n          , Sanaman, G. and Rai, N. (2008), \u201cFederated search: new option for libraries in the digital era\u201d,                   International CALIBER                , pp. 267-285."},{"issue":"3\/4","key":"2025072816225140800_b19","first-page":"253","article-title":"Librarian perspectives on teaching metasearch and federated search technologies","volume":"12","author":"","year":"2007","journal-title":"Internet Reference Services Quarterly"},{"issue":"2","key":"2025072816225140800_b20","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1007\/s11704-012-2859-3","article-title":"Stratified sampling for data mining on the deep web","volume":"6","author":"","year":"2012","journal-title":"Frontiers of Computer Science"},{"key":"2025072816225140800_b21","doi-asserted-by":"crossref","unstructured":"Lu, J.\n           and Callan, J. (2005), \u201cFederated search of text-based digital libraries in hierarchical peer-to-peer networks\u201d,                   Advances in Information Retrieval                , Springer, Berlin Heidelberg, pp. 52-66.","DOI":"10.1007\/978-3-540-31865-1_5"},{"key":"2025072816225140800_b22","unstructured":"Mohamed, K.\n           (2006),                   Merging Multiple Search Results Approach for Meta-Search Engines                ,                   PhD thesis                , University of Pittsburgh, Pittsburgh, PA."},{"issue":"4","key":"2025072816225140800_b23","doi-asserted-by":"crossref","first-page":"418","DOI":"10.1108\/00330330810912098","article-title":"Web usage mining analysis of federated search tools for Egyptian scholars","volume":"42","author":"","year":"2008","journal-title":"Program: Electronic Library and Information Systems"},{"issue":"3","key":"2025072816225140800_b24","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/2.121503","article-title":"The usability engineering life cycle","volume":"25","author":"","year":"1992","journal-title":"Computer"},{"issue":"3","key":"2025072816225140800_b25","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1080\/00987913.2006.10765056","article-title":"Federated searching and usability testing: building the perfect beast","volume":"32","author":"","year":"2006","journal-title":"Serials Review"},{"key":"2025072816225140800_b26","doi-asserted-by":"crossref","unstructured":"Si, L.\n           and Callan, J. (2005), \u201cModeling search engine effectiveness for federated search\u201d, Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval,                   ACM                , pp. 83-90.","DOI":"10.1145\/1076034.1076051"},{"key":"2025072816225140800_b27","doi-asserted-by":"crossref","unstructured":"Tchangalova, N.\n           and Stilwell, F. (2012), \u201cSearch engines and beyond: a toolkit for finding free online resources for science, technology, and engineering\u201d,                   Science and Technology Librarianship                , available at: www.istl.org\/12-spring\/internet1.html (accessed 7 July 2013).","DOI":"10.29173\/istl1554"},{"key":"2025072816225140800_b28","unstructured":"Tennant, R.\n           (2001),                   Digital Libraries: Cross-Database Search: One-Stop Shopping                , available at: http:\/\/libraryjournal.reviewsnews.com\/index.asp?layout=articlePrint&articleID=CA170458 (accessed 7 July 2013)."},{"key":"2025072816225140800_b29","doi-asserted-by":"crossref","unstructured":"Tsikrika, T.\n           and Lalmas, M. (2001), \u201cMerging techniques for performing data fusion on the web\u201d, Proceedings of the Tenth International Conference on Information and Knowledge Management,                   ACM                , pp. 127-134.","DOI":"10.1145\/502585.502608"},{"issue":"5","key":"2025072816225140800_b30","doi-asserted-by":"crossref","first-page":"697","DOI":"10.1016\/S0306-4573(00)00010-8","article-title":"Variations in relevance judgments and the measurement of retrieval effectiveness","volume":"36","author":"","year":"2000","journal-title":"Information Processing & Management"},{"issue":"4","key":"2025072816225140800_b31","doi-asserted-by":"crossref","first-page":"292","DOI":"10.5860\/crl.68.4.292","article-title":"Usability testing of a metasearch interface: a case study","volume":"68","author":"","year":"2007","journal-title":"College & Research Libraries"},{"issue":"4","key":"2025072816225140800_b32","doi-asserted-by":"crossref","first-page":"548","DOI":"10.1109\/69.536248","article-title":"WISE: a World Wide Web resource database system","volume":"8","author":"","year":"1996","journal-title":"IEEE Transactions on Knowledge and Data Engineering"}],"container-title":["The Electronic Library"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/EL-12-2013-0211","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/EL-12-2013-0211\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/el\/article-pdf\/33\/6\/1079\/229115\/el-12-2013-0211.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/www.emerald.com\/el\/article-pdf\/33\/6\/1079\/229115\/el-12-2013-0211.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,28]],"date-time":"2025-07-28T20:23:07Z","timestamp":1753734187000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.emerald.com\/el\/article\/33\/6\/1079\/39015\/Evaluating-federated-search-tools-usability-and"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,11,2]]},"references-count":32,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2015,11,2]]}},"URL":"https:\/\/doi.org\/10.1108\/el-12-2013-0211","relation":{},"ISSN":["0264-0473","1758-616X"],"issn-type":[{"type":"print","value":"0264-0473"},{"type":"electronic","value":"1758-616X"}],"subject":[],"published":{"date-parts":[[2015,11,2]]}}}