{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T00:47:26Z","timestamp":1790815646727,"version":"4.1.0"},"reference-count":47,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2019,9,13]],"date-time":"2019-09-13T00:00:00Z","timestamp":1568332800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Data Science"],"published-print":{"date-parts":[[2020,6,12]]},"abstract":"<jats:p>An increasing number of solutions aim to support the steady increase of the number of requirements and requests for Linked Data at scale. This plethora of solutions leads to a growing need for objective means that facilitate the selection of adequate solutions for particular use cases. We hence present Hobbit, a distributed benchmarking platform designed for the unified execution of benchmarks for Linked Data solutions. The Hobbit benchmarking platform is based on the FAIR principles and is the first benchmarking platform able to scale up to benchmarking real-world scenarios for Big Linked Data solutions. Our online instance of the platform has more than 300 registered users and offers more than 40 benchmarks. It has been used in eleven benchmarking challenges and for more than 13000 experiments. We give an overview of the results achieved during 2 of these challenges and point to some of the novel insights that were gained from the results of the platform. Hobbit is open-source and available at http:\/\/github.com\/hobbit-project .<\/jats:p>","DOI":"10.3233\/ds-190021","type":"journal-article","created":{"date-parts":[[2019,9,13]],"date-time":"2019-09-13T11:42:36Z","timestamp":1568374956000},"page":"15-35","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":14,"title":["HOBBIT: A\u00a0platform for benchmarking Big Linked Data"],"prefix":"10.1177","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8609-8277","authenticated-orcid":false,"given":"Michael","family":"R\u00f6der","sequence":"first","affiliation":[{"name":"Data Science Department, University of Paderborn,Germany. E-mail:\u00a0"},{"name":"Institute for Applied Informatics, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9637-6197","authenticated-orcid":false,"given":"Denis","family":"Kuchelev","sequence":"additional","affiliation":[{"name":"Institute for Applied Informatics, Germany"},{"name":"Data Science Department, University of Paderborn,Germany."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7112-3516","authenticated-orcid":false,"given":"Axel-Cyrille","family":"Ngonga Ngomo","sequence":"additional","affiliation":[{"name":"Institute for Applied Informatics, Germany"},{"name":"Data Science Department, University of Paderborn,Germany. E-mail:\u00a0"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2019,9,13]]},"reference":[{"key":"ref001","unstructured":"A.\u00a0Algergawy, M.\u00a0Cheatham, D.\u00a0Faria, A.\u00a0Ferrara, I.\u00a0Fundulaki, I.\u00a0Harrow, S.\u00a0Hertling, E.\u00a0Jim\u00e9nez-Ruiz, N.\u00a0Karam, A.\u00a0Khiat, P.\u00a0Lambrix, H.\u00a0Li, S.\u00a0Montanelli, H.\u00a0Paulheim, C.\u00a0Pesquita, T.\u00a0Saveta, D.\u00a0Schmidt, P.\u00a0Shvaiko, A.\u00a0Splendiani, E.\u00a0Thi\u00e9blin, C.\u00a0Trojahn, J.\u00a0Vata\u0161\u010dinov\u00e1, O.\u00a0Zamazal and L.\u00a0Zhou, Results of the ontology alignment evaluation initiative 2018, in: Proceedings of the 13th International Workshop on Ontology Matching (OM 2018), Monterey, CA, USA, October 8, 2018, P.\u00a0Shvaiko, J.\u00a0Euzenat, E.\u00a0Jim\u00e9nez-Ruiz, M.\u00a0Cheatham and O.\u00a0Hassanzadeh, eds, CEUR-WS, 2018, pp.\u00a076\u2013116, http:\/\/ceur-ws.org\/Vol-2288\/oaei18_paper0.pdf."},{"key":"ref002","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-23032-5_1"},{"key":"ref003","unstructured":"J.\u00a0Baldridge, The opennlp project, 2005, http:\/\/opennlp.apache.org\/index.html."},{"key":"ref004","doi-asserted-by":"publisher","DOI":"10.4018\/jswis.2009040101"},{"key":"ref005","unstructured":"D.\u00a0Brickley, R.V.\u00a0Guha and B.\u00a0McBride, RDF schema 1.1,\n                      W3C Working Group Note\n                      , 2014, http:\/\/www.w3.org\/TR\/2014\/REC-rdf-schema-20140225\/."},{"key":"ref006","doi-asserted-by":"publisher","DOI":"10.1145\/2701583.2701591"},{"key":"ref007","unstructured":"J.J.\u00a0Carroll and J.Z.\u00a0Pan, XML schema datatypes in RDF and OWL,\n                      W3C Working Group Note\n                      , 2006, http:\/\/www.w3.org\/TR\/2006\/NOTE-swbp-xsch-datatypes-20060314\/."},{"key":"ref008","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-21569-3"},{"key":"ref009","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2017.8258249"},{"key":"ref010","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-68204-4_5"},{"key":"ref011","doi-asserted-by":"publisher","DOI":"10.13140\/2.1.4942.9766"},{"key":"ref012","unstructured":"R.\u00a0Cyganiak and D.\u00a0Reynolds, The RDF data cube vocabulary,\n                      W3C Recommendation\n                      , 2014, http:\/\/www.w3.org\/TR\/2014\/REC-vocab-data-cube-20140116\/."},{"key":"ref013","doi-asserted-by":"publisher","DOI":"10.1016\/j.websem.2016.06.002"},{"key":"ref014","doi-asserted-by":"publisher","DOI":"10.14778\/2732240.2732246"},{"key":"ref015","doi-asserted-by":"publisher","DOI":"10.3115\/1219840.1219885"},{"key":"ref016","unstructured":"I.\u00a0Fundulaki, Deliverable 1.2.1: Requirements specification from the community, 2016, http:\/\/project-hobbit.eu\/about\/deliverables\/."},{"key":"ref017","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-00072-1_1"},{"key":"ref018","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-69146-6_1"},{"key":"ref019","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2463712"},{"key":"ref020","unstructured":"J.\u00a0Gray and C.\u00a0Levine, Thousands of debitcredit transactions-per-second: Easy and inexpensive, arXiv preprint cs\/0701161, 2007, https:\/\/arxiv.org\/abs\/cs\/0701161."},{"key":"ref021","doi-asserted-by":"publisher","DOI":"10.1145\/3093742.3096342"},{"key":"ref022","doi-asserted-by":"publisher","DOI":"10.1145\/3210284.3220510"},{"key":"ref023","doi-asserted-by":"publisher","DOI":"10.1016\/j.websem.2005.06.005"},{"key":"ref024","unstructured":"E.\u00a0Jim\u00e9nez-Ruiz, T.\u00a0Saveta, O.\u00a0Zamazal, S.\u00a0Hertling, M.\u00a0Roder, I.\u00a0Fundulaki, A.\u00a0Ngonga Ngomo, M.\u00a0Sherif, A.\u00a0Annane, Z.\u00a0Bellahsene, S.\u00a0Ben Yahia, G.\u00a0Diallo, D.\u00a0Faria, M.\u00a0Kachroudi, A.\u00a0Khiat, P.\u00a0Lambrix, H.\u00a0Li, M.\u00a0Mackeprang, M.\u00a0Mohammadi, M.\u00a0Rybinski, B.S.\u00a0Balasubramani and C.\u00a0Trojahn, Introducing the HOBBIT platform into the ontology alignment evaluation campaign, in: Proceedings of the 13th International Workshop on Ontology Matching (OM 2018), Monterey, CA, USA, October 8, 2018, P.\u00a0Shvaiko, J.\u00a0Euzenat, E.\u00a0Jim\u00e9nez-Ruiz, M.\u00a0Cheatham and O.\u00a0Hassanzadeh, eds, CEUR-WS, 2018, pp.\u00a049\u201360, http:\/\/ceur-ws.org\/Vol-2288\/om2018_LTpaper5.pdf."},{"key":"ref025","unstructured":"T.\u00a0Lebo, S.\u00a0Sahoo and D.\u00a0McGuinness, PROV-O: The PROV ontology,\n                      W3C Recommendation\n                      , 2013, http:\/\/www.w3.org\/TR\/2013\/REC-prov-o-20130430\/."},{"key":"ref026","unstructured":"F.\u00a0Manola, E.\u00a0Miller and B.\u00a0McBride, RDF 1.1 primer,\n                      W3C Working Group Note\n                      , 2014, https:\/\/www.w3.org\/TR\/2014\/NOTE-rdf11-primer-20140624\/."},{"key":"ref027","doi-asserted-by":"publisher","DOI":"10.1145\/2063518.2063519"},{"key":"ref028","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25073-6_29"},{"key":"ref029","doi-asserted-by":"publisher","DOI":"10.1145\/3148011.3148024"},{"key":"ref030","unstructured":"D.\u00a0Nadeau, Balie\u2014baseline information extraction: Multilingual information extraction from text with machine learning and natural language techniques, Technical report, University of Ottawa, 2005, http:\/\/balie.sourceforge.net\/dnadeau05balie.pdf."},{"key":"ref031","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-00072-1_6"},{"key":"ref032","doi-asserted-by":"publisher","DOI":"10.3233\/SW-150210"},{"key":"ref033","doi-asserted-by":"publisher","DOI":"10.3115\/1596374.1596399"},{"key":"ref034","unstructured":"M.\u00a0R\u00f6der, T.\u00a0Saveta, I.\u00a0Fundulaki and A.C.N.\u00a0Ngomo, HOBBIT link discovery benchmarks at ontology matching 2017, in: Proceedings of the 12th International Workshop on Ontology Matching (OM-2017), Vienna, Austria, October 21, 2017, P.\u00a0Shvaiko, J.\u00a0Euzenat, E.\u00a0Jim\u00e9nez-Ruiz, M.\u00a0Cheatham and O.\u00a0Hassanzadeh, eds, CEUR-WS, 2017, pp.\u00a0209\u2013210, http:\/\/ceur-ws.org\/Vol-2032\/om2017_poster2.pdf."},{"key":"ref035","doi-asserted-by":"publisher","DOI":"10.3233\/SW-170286"},{"key":"ref036","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-25010-6_15"},{"key":"ref037","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-25007-6_4"},{"key":"ref038","doi-asserted-by":"publisher","DOI":"10.3115\/1119176.1119195"},{"key":"ref039","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-11964-9_33"},{"key":"ref040","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-00072-1_4"},{"key":"ref041","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-69146-6_4"},{"key":"ref042","doi-asserted-by":"publisher","DOI":"10.3115\/1072017.1072023"},{"key":"ref043","unstructured":"G.\u00a0Tsatsaronis, M.\u00a0Schroeder, G.\u00a0Paliouras, Y.\u00a0Almirantis, I.\u00a0Androutsopoulos, E.\u00a0Gaussier, P.\u00a0Gallinari, T.\u00a0Artieres, M.R.\u00a0Alvers, M.\u00a0Zschunkeet al., BioASQ: A challenge on large-scale biomedical semantic indexing and question answering, in: AAAI Fall Symposium: Information Retrieval and Knowledge Discovery in Biomedical Text, 2012, https:\/\/www.aaai.org\/ocs\/index.php\/FSS\/FSS12\/paper\/viewPaper\/5600."},{"key":"ref044","unstructured":"C.\u00a0Unger, C.\u00a0Forascu, V.\u00a0Lopez, A.C.N.\u00a0Ngomo, E.\u00a0Cabrio, P.\u00a0Cimiano and S.\u00a0Walter, Question answering over linked data (QALD-4), in: Working Notes for CLEF 2014 Conference, 2014, http:\/\/www.ceur-ws.org\/Vol-1180\/CLEF2014wn-QA-UngerEt2014.pdf."},{"key":"ref045","unstructured":"C.\u00a0Unger, C.\u00a0Forascu, V.\u00a0Lopez, A.N.\u00a0Ngomo, E.\u00a0Cabrio, P.\u00a0Cimiano and S.\u00a0Walter, Question answering over linked data (QALD-5), in: CLEF, 2015, http:\/\/ceur-ws.org\/Vol-1391\/173-CR.pdf."},{"key":"ref046","doi-asserted-by":"publisher","DOI":"10.3233\/SW-180312"},{"key":"ref047","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2016.18"}],"container-title":["Data Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/DS-190021","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.3233\/DS-190021","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/DS-190021","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T18:10:03Z","timestamp":1777399803000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.3233\/DS-190021"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9,13]]},"references-count":47,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,6,12]]}},"alternative-id":["10.3233\/DS-190021"],"URL":"https:\/\/doi.org\/10.3233\/ds-190021","relation":{},"ISSN":["2451-8484","2451-8492"],"issn-type":[{"value":"2451-8484","type":"print"},{"value":"2451-8492","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,9,13]]}}}