{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T10:18:56Z","timestamp":1777889936613,"version":"3.51.4"},"reference-count":49,"publisher":"SAGE Publications","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SW"],"published-print":{"date-parts":[[2021,10,4]]},"abstract":"<jats:p>Finding a good query plan is key to the optimization of query runtime. This holds in particular for cost-based federation engines, which make use of cardinality estimations to achieve this goal. A number of studies compare SPARQL federation engines across different performance metrics, including query runtime, result set completeness and correctness, number of sources selected and number of requests sent. Albeit informative, these metrics are generic and unable to quantify and evaluate the accuracy of the cardinality estimators of cost-based federation engines. To thoroughly evaluate cost-based federation engines, the effect of estimated cardinality errors on the overall query runtime performance must be measured. In this paper, we address this challenge by presenting novel evaluation metrics targeted at a fine-grained benchmarking of cost-based federated SPARQL query engines. We evaluate five cost-based federated SPARQL query engines using existing as well as novel evaluation metrics by using LargeRDFBench queries. Our results provide a detailed analysis of the experimental outcomes that reveal novel insights, useful for the development of future cost-based federated SPARQL query processing engines.<\/jats:p>","DOI":"10.3233\/sw-200420","type":"journal-article","created":{"date-parts":[[2021,1,22]],"date-time":"2021-01-22T17:27:04Z","timestamp":1611336424000},"page":"843-868","source":"Crossref","is-referenced-by-count":7,"title":["An empirical evaluation of cost-based federated SPARQL query processing engines"],"prefix":"10.1177","volume":"12","author":[{"given":"Umair","family":"Qudus","sequence":"first","affiliation":[{"name":"DKE, Kyung Hee University, South Korea. E-mails:\u00a0umair.qudus@khu.ac.kr,\u00a0yklee@khu.ac.kr"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Muhammad","family":"Saleem","sequence":"additional","affiliation":[{"name":"AKSW, Leipzig, Germany. E-mail:\u00a0saleem@informatik.uni-leipzig.de"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Axel-Cyrille","family":"Ngonga Ngomo","sequence":"additional","affiliation":[{"name":"University of Paderborn, Germany. E-mail:\u00a0axel.ngonga@upb.de"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Young-Koo","family":"Lee","sequence":"additional","affiliation":[{"name":"DKE, Kyung Hee University, South Korea. E-mails:\u00a0umair.qudus@khu.ac.kr,\u00a0yklee@khu.ac.kr"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"issue":"4","key":"10.3233\/SW-200420_ref1","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1145\/3186728.3164144","article-title":"A system for querying linked data at scale","volume":"11","author":"Abdelaziz","year":"2017","journal-title":"Proc. VLDB Endow."},{"key":"10.3233\/SW-200420_ref2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25073-6_2"},{"key":"10.3233\/SW-200420_ref3","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-68204-4_1"},{"key":"10.3233\/SW-200420_ref4","doi-asserted-by":"publisher","DOI":"10.1145\/2684200.2684321"},{"key":"10.3233\/SW-200420_ref6","unstructured":"K.\u00a0Alexander, R.\u00a0Cyganiak, M.\u00a0Hausenblas and J.\u00a0Zhao, Describing linked datasets \u2013 On the design and usage of void, the vocabulary of interlinked datasets, in: Linked Data on the Web Workshop (LDOW 09), in Conjunction with 18th International World Wide Web Conference (WWW 09), Vol.\u00a0538, 2010."},{"key":"10.3233\/SW-200420_ref7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.4018\/jswis.2009040101","article-title":"The Berlin SPARQL benchmark","volume":"5","author":"Bizer","year":"2009","journal-title":"International Journal on Semantic Web and Information Systems (IJSWIS)"},{"key":"10.3233\/SW-200420_ref8","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-41338-4_18"},{"key":"10.3233\/SW-200420_ref9","doi-asserted-by":"publisher","DOI":"10.1145\/2814864.2814886"},{"key":"10.3233\/SW-200420_ref10","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-68204-4_5"},{"key":"10.3233\/SW-200420_ref11","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-29038-1_12"},{"key":"10.3233\/SW-200420_ref12","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-64468-4_1"},{"key":"10.3233\/SW-200420_ref13","unstructured":"O.\u00a0G\u00f6rlitz and S.\u00a0Staab, SPLENDID: SPARQL endpoint federation exploiting VOID descriptions, in: Proceedings of the Second International Conference on Consuming Linked Data (COLD \u201911), Vol.\u00a0782, CEUR-WS.org, Aachen, Germany, 2010, pp.\u00a013\u201324."},{"key":"10.3233\/SW-200420_ref14","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-35176-1_8"},{"key":"10.3233\/SW-200420_ref15","unstructured":"A.\u00a0Gubichev and T.\u00a0Neumann, Exploiting the query structure for efficient join ordering in SPARQL queries, in: EDBT, Vol.\u00a014, 2014, pp.\u00a0439\u2013450."},{"key":"10.3233\/SW-200420_ref16","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04930-9_19"},{"key":"10.3233\/SW-200420_ref17","unstructured":"A.\u00a0Hasnain, R.\u00a0Fox, S.\u00a0Decker and H.F.\u00a0Deus, Cataloguing and linking life sciences LOD cloud, in: 1st International Workshop on Ontology Engineering in a Data-Driven World (OEDW 2012) Collocated with 8th International Conference on Knowledge Engineering and Knowledge Management (EKAW 2012), 2012, pp.\u00a0114\u2013130."},{"issue":"1","key":"10.3233\/SW-200420_ref18","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1186\/s13326-016-0111-z","article-title":"BioFed: Federated query processing over life sciences linked open data","volume":"8","author":"Hasnain","year":"2017","journal-title":"Journal of Biomedical Semantics"},{"key":"10.3233\/SW-200420_ref19","doi-asserted-by":"publisher","DOI":"10.3233\/978-1-61499-894-5-203"},{"key":"10.3233\/SW-200420_ref20","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-15615-6_8"},{"issue":"9","key":"10.3233\/SW-200420_ref21","doi-asserted-by":"publisher","first-page":"813","DOI":"10.1080\/03610927708827533","article-title":"Robust regression using iteratively reweighted least-squares","volume":"6","author":"Holland","year":"1977","journal-title":"Communications in Statistics \u2013 Theory and Methods"},{"key":"10.3233\/SW-200420_ref22","doi-asserted-by":"publisher","first-page":"492","DOI":"10.1007\/978-1-4612-4380-9_35","volume-title":"Robust Estimation of a Location Parameter","author":"Huber","year":"1992"},{"key":"10.3233\/SW-200420_ref23","doi-asserted-by":"publisher","DOI":"10.13140\/2.1.3153.9204"},{"issue":"1","key":"10.3233\/SW-200420_ref24","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1186\/s13326-017-0112-6","article-title":"SAFE: SPARQL federation over RDF data cubes with access control","volume":"8","author":"Khan","year":"2017","journal-title":"Journal of biomedical semantics"},{"issue":"4","key":"10.3233\/SW-200420_ref25","doi-asserted-by":"publisher","first-page":"422","DOI":"10.1145\/371578.371598","article-title":"The state of the art in distributed query processing","volume":"32","author":"Kossmann","year":"2000","journal-title":"ACM Comput. Surv."},{"key":"10.3233\/SW-200420_ref26","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-21034-1_10"},{"issue":"3","key":"10.3233\/SW-200420_ref27","doi-asserted-by":"publisher","first-page":"204","DOI":"10.14778\/2850583.2850594","article-title":"How good are query optimizers, really?","volume":"9","author":"Leis","year":"2015","journal-title":"Proc. VLDB Endow."},{"key":"10.3233\/SW-200420_ref28","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25106-1_28"},{"issue":"1","key":"10.3233\/SW-200420_ref29","doi-asserted-by":"publisher","first-page":"982","DOI":"10.14778\/1687627.1687738","article-title":"Preventing bad plans by bounding the impact of cardinality estimation errors","volume":"2","author":"Moerkotte","year":"2009","journal-title":"Proc. VLDB Endow."},{"key":"10.3233\/SW-200420_ref30","doi-asserted-by":"publisher","first-page":"471","DOI":"10.1007\/978-3-319-68288-4_28","article-title":"The odyssey approach for optimizing federated SPARQL queries","volume":"2017","author":"Montoya","year":"2017","journal-title":"The Semantic Web \u2013 ISWC"},{"key":"10.3233\/SW-200420_ref31","doi-asserted-by":"publisher","DOI":"10.5555\/2887367.2887373"},{"key":"10.3233\/SW-200420_ref32","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-35173-0_21"},{"key":"10.3233\/SW-200420_ref33","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25073-6_29"},{"key":"10.3233\/SW-200420_ref34","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2011.5767868"},{"issue":"3","key":"10.3233\/SW-200420_ref35","doi-asserted-by":"publisher","first-page":"466","DOI":"10.1137\/0611032","article-title":"Robust regression computation using iteratively reweighted least squares","volume":"11","author":"O\u2019Leary","year":"1990","journal-title":"SIAM J. Matrix Anal. Appl."},{"key":"10.3233\/SW-200420_ref36","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-68234-9_39"},{"key":"10.3233\/SW-200420_ref37","doi-asserted-by":"publisher","DOI":"10.1002\/0471725382"},{"key":"10.3233\/SW-200420_ref38","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1016\/j.websem.2017.12.005","article-title":"LargeRDFBench: A billion triples benchmark for SPARQL endpoint federation","volume":"48","author":"Saleem","year":"2018","journal-title":"Journal of Web Semantics"},{"issue":"5","key":"10.3233\/SW-200420_ref39","doi-asserted-by":"publisher","first-page":"493","DOI":"10.3233\/SW-150186","article-title":"A fine-grained evaluation of SPARQL endpoint federation systems","volume":"7","author":"Saleem","year":"2016","journal-title":"Semantic Web Journal"},{"key":"10.3233\/SW-200420_ref40","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-07443-6_13"},{"key":"10.3233\/SW-200420_ref41","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-41335-3_36"},{"key":"10.3233\/SW-200420_ref42","doi-asserted-by":"publisher","DOI":"10.1186\/2041-1480-5-47"},{"key":"10.3233\/SW-200420_ref43","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2018.09.016"},{"key":"10.3233\/SW-200420_ref44","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313556"},{"key":"10.3233\/SW-200420_ref45","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25073-6_37"},{"key":"10.3233\/SW-200420_ref46","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2009.28"},{"key":"10.3233\/SW-200420_ref47","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25073-6_38"},{"issue":"6","key":"10.3233\/SW-200420_ref48","doi-asserted-by":"publisher","first-page":"585","DOI":"10.3233\/SW-140164","article-title":"Link traversal querying for a diverse web of data","volume":"6","author":"Umbrich","year":"2015","journal-title":"Semantic Web Journal"},{"key":"10.3233\/SW-200420_ref49","unstructured":"X.\u00a0Wang, T.\u00a0Tiropanis and H.\u00a0Davis, LHD optimising: Linked data query processing using parallelisation, in: Workshop on Linked Data on the Web (LDOW \u201913), Proceedings of the WWW 2013, CEUR Workshop Proceedings, Vol.\u00a0996, CEUR-WS.org, Rio de Janeiro, Brazil, 2013."},{"issue":"4","key":"10.3233\/SW-200420_ref50","doi-asserted-by":"publisher","first-page":"84:1","DOI":"10.1145\/3177850","article-title":"RDF data storage and query processing schemes: A survey","volume":"51","author":"Wylot","year":"2018","journal-title":"ACM Comput. Surv."}],"container-title":["Semantic Web"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/SW-200420","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T05:25:13Z","timestamp":1777613113000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/SW-200420"}},"subtitle":[],"editor":[{"given":"Ruben","family":"Verborgh","sequence":"additional","affiliation":[{"name":"Ghent University, Belgium"}],"role":[{"role":"editor","vocabulary":"crossref"}]},{"given":"Muhammad","family":"Saleem","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]},{"given":"Ruben","family":"Verborgh","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]},{"given":"Muhammad Intizar","family":"Ali","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]},{"given":"Olaf","family":"Hartig","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2021,10,4]]},"references-count":49,"journal-issue":{"issue":"6"},"URL":"https:\/\/doi.org\/10.3233\/sw-200420","relation":{},"ISSN":["2210-4968","1570-0844"],"issn-type":[{"value":"2210-4968","type":"electronic"},{"value":"1570-0844","type":"print"}],"subject":[],"published":{"date-parts":[[2021,10,4]]}}}