{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T18:18:03Z","timestamp":1790619483519,"version":"4.1.0"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2025,12,4]]},"abstract":"<jats:p>Knowledge graphs and ontologies play an essential role in integrating, standardizing, and reasoning about complex data across domains. In recent studies, leveraging knowledge graphs in AI use cases, instead of traditional relational databases, led to quality improvements by up to 38 percentage points. However, learning ontologies from relational databases remains a challenging task due to the impedance mismatch between both modeling concepts. An understanding of which ontology learning system performs best, and why, is missing, as no established benchmark exists.<\/jats:p>\n                  <jats:p>We present BURR, a benchmark for evaluating ontology learning systems from relational databases. To evaluate the ontology learning space, we introduce a novel mapping-based metric and provide a comprehensive benchmark data collection. This collection of 54 scenarios consists of real-world database-ontology mappings, including industry data, and of a micro-benchmark evaluating the behavior of systems in encapsulated scenarios. We demonstrate the applicability of BURR by evaluating widely used ontology learning systems, including traditional rule-based as well as LLM-based approaches, on the benchmark. The results emphasize the current strengths of simple rule-based approaches compared to LLM-based systems, while also highlighting the significant research potential of LLMs in ontology learning.<\/jats:p>","DOI":"10.1145\/3769770","type":"journal-article","created":{"date-parts":[[2025,12,6]],"date-time":"2025-12-06T04:32:13Z","timestamp":1764995533000},"page":"1-27","source":"Crossref","is-referenced-by-count":3,"title":["<scp>Burr:<\/scp>\n                    A Benchmark for Ontology Learning from Relational Databases"],"prefix":"10.1145","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7532-9925","authenticated-orcid":false,"given":"Lukas","family":"Laskowski","sequence":"first","affiliation":[{"name":"Hasso Plattner Institute, University of Potsdam, Potsdam, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2204-3138","authenticated-orcid":false,"given":"Michael","family":"Hladik","sequence":"additional","affiliation":[{"name":"SAP SE, Walldorf, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5420-0663","authenticated-orcid":false,"given":"Jan","family":"Portisch","sequence":"additional","affiliation":[{"name":"SAP SE, Walldorf, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0675-4116","authenticated-orcid":false,"given":"Fabian","family":"Panse","sequence":"additional","affiliation":[{"name":"University of Augsburg, Augsburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4483-1389","authenticated-orcid":false,"given":"Felix","family":"Naumann","sequence":"additional","affiliation":[{"name":"Hasso Plattner Institute, University of Potsdam, Potsdam, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,12,5]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Central Intelligence Agency. 2024. The World Factbook. https:\/\/www.cia.gov\/the-world-factbook\/. Accessed: 2024-12-30."},{"key":"e_1_2_2_2_1","unstructured":"AI Data Analytics Network and Elliot Leavy. 2022. Data Quality Crisis: New Survey Reveals 77% of Organizations Have Quality Issues. https:\/\/www.aidataanalytics.network\/data-governance\/articles\/data-quality-crisis-new-survey-reveals-77-of-organizations-have-quality-issues Accessed: 2024-11-27."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/SKG.2011.39"},{"key":"e_1_2_2_4_1","volume-title":"Proceedings of the International Semantic Web Conference (ISWC). Springer, 408-427","author":"Giglou Hamed Babaei","year":"2023","unstructured":"Hamed Babaei Giglou, Jennifer D'Souza, and S\u00f6ren Auer. 2023. LLMs4OL: Large Language Models for Ontology Learning. In Proceedings of the International Semantic Web Conference (ISWC). Springer, 408-427."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/27633.27634"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-021-00412-2"},{"key":"e_1_2_2_7_1","unstructured":"Bilal Benmammar. 2024. FromRDBtoOnto: A framework for transforming relational database schemas into ontologies. https:\/\/github.com\/bilalbenma\/FromRDBtoOnto Accessed: 2024-12-29."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.14778\/3407790.3407824"},{"key":"e_1_2_2_9_1","volume-title":"D2RQ-Treating non-RDF databases as virtual RDF graphs. World Wide Web Internet and Web Information Systems (01","author":"Bizer Christian","year":"2005","unstructured":"Christian Bizer and Andy Seaborne. 2005. D2RQ-Treating non-RDF databases as virtual RDF graphs. World Wide Web Internet and Web Information Systems (01 2005)."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.datak.2023.102157"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/362384.362685"},{"key":"e_1_2_2_12_1","volume-title":"California","author":"Codd E. F.","year":"1971","unstructured":"E. F. Codd. 1971. Further Normalization of the Data Base Relational Model. Research Report \/ RJ \/ IBM \/ San Jose, California, Vol. RJ909 (1971)."},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2507.06261"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.3233\/SW-160217"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/COGMI48466.2019.00029"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-19890-3_21"},{"key":"e_1_2_2_17_1","article-title":"Statistical Comparisons of Classifiers over Multiple Data Sets","volume":"7","author":"Dem\u0161ar Janez","year":"2006","unstructured":"Janez Dem\u0161ar. 2006. Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research (JMLR), Vol. 7 (Dec. 2006), 1-30.","journal-title":"Journal of Machine Learning Research (JMLR)"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/3430915.3442430"},{"key":"e_1_2_2_19_1","volume-title":"Proceedings of the Workshop on Linked Data on the Web co-located with the International World Wide Web Conference (LDOW@WWW) (CEUR Workshop Proceedings)","volume":"1184","author":"Dimou Anastasia","year":"2014","unstructured":"Anastasia Dimou, Miel Vander Sande, Pieter Colpaert, Ruben Verborgh, Erik Mannens, and Rik Van de Walle. 2014a. RML: A Generic Language for Integrated RDF Mappings of Heterogeneous Data. In Proceedings of the Workshop on Linked Data on the Web co-located with the International World Wide Web Conference (LDOW@WWW) (CEUR Workshop Proceedings), Vol. 1184. https:\/\/ceur-ws.org\/Vol-1184\/ldow2014_paper_01.pdf"},{"key":"e_1_2_2_20_1","volume-title":"Proceedings of the Workshop on Linked Data on the Web co-located with the International World Wide Web Conference (LDOW@WWW) (CEUR Workshop Proceedings)","volume":"1184","author":"Dimou Anastasia","year":"2014","unstructured":"Anastasia Dimou, Miel Vander Sande, Pieter Colpaert, Ruben Verborgh, Erik Mannens, and Rik Van de Walle. 2014b. RML: A Generic Language for Integrated RDF Mappings of Heterogeneous Data. In Proceedings of the Workshop on Linked Data on the Web co-located with the International World Wide Web Conference (LDOW@WWW) (CEUR Workshop Proceedings), Vol. 1184. CEUR-WS.org. https:\/\/ceur-ws.org\/Vol-1184\/ldow2014_paper_01.pdf"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-13881-2_29"},{"key":"e_1_2_2_22_1","article-title":"The Database Normalization Theory and the Theory of Normalized Systems: Finding a Common Ground","volume":"4","author":"Eessaar Erki","year":"2016","unstructured":"Erki Eessaar. 2016. The Database Normalization Theory and the Theory of Normalized Systems: Finding a Common Ground. Baltic Journal of Modern Computing, Vol. 4, 1 (2016).","journal-title":"Baltic Journal of Modern Computing"},{"key":"e_1_2_2_23_1","volume-title":"Navathe","author":"Elmasri Ramez","year":"2010","unstructured":"Ramez Elmasri and Shamkant B. Navathe. 2010. Fundamentals of Database Systems, 6th Edition. Addison-Wesley-Longman.","edition":"6"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-38721-0"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-38721-0"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1937.10503522"},{"key":"e_1_2_2_27_1","volume-title":"Database Systems: The Complete Book (2 ed.)","author":"Garcia-Molina Hector","year":"2008","unstructured":"Hector Garcia-Molina, Jeffrey D. Ullman, and Jennifer Widom. 2008. Database Systems: The Complete Book (2 ed.). Prentice Hall Press, USA."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447772"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-25010-6_7"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-62466-8_29"},{"key":"e_1_2_2_31_1","volume-title":"Optique: Towards OBDA Systems for Industry. In The Semantic Web: ESWC Satellite Events","author":"Kharlamov Evgeny","year":"2013","unstructured":"Evgeny Kharlamov, Ernesto Jim\u00e9nez-Ruiz, Dmitriy Zheleznyakov, Dimitris Bilidas, Martin Giese, Peter Haase, Ian Horrocks, Herald Kllapi, Manolis Koubarakis, \u00d6zg\u00fcr \u00d6z\u00e7ep, Mariano Rodr\u00edguez-Muro, Riccardo Rosati, Michael Schmidt, Rudolf Schlatte, Ahmet Soylu, and Arild Waaler. 2013. Optique: Towards OBDA Systems for Industry. In The Semantic Web: ESWC Satellite Events. Springer, 125-140."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.websem.2022.100761"},{"key":"e_1_2_2_33_1","unstructured":"Noah MacCallum and Julian Lee. 2025. GPT-4.1 Prompting Guide. OpenAI Cookbook. https:\/\/cookbook.openai.com\/examples\/gpt4-1_prompting_guide."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4615-0925-7_7"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/5254.920602"},{"key":"e_1_2_2_36_1","unstructured":"Wolfgang May. 1999. Information extraction and integration with Florid: The Mondial case study. Technical Report."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/J.IS.2025.102549"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12652-023-04693-8"},{"key":"e_1_2_2_39_1","unstructured":"Peter Bj\u00f6rn Nemenyi. 1963. Distribution-free Multiple Comparisons. Ph.D. Dissertation. Princeton University."},{"key":"e_1_2_2_40_1","unstructured":"OpenAI. 2024. GPT-4o Technical Report. https:\/\/openai.com\/research\/gpt-4o Accessed: 2025-03-31."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2303.08774"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-17449-0_2"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.5441\/002\/EDBT.2017.31"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-18818-8_2"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.4018\/ijswis.2014040102"},{"key":"e_1_2_2_46_1","unstructured":"PricewaterhouseCoopers. 2017. Sizing the Prize: What's the Real Value of AI for Your Business and How Can You Capitalise? https:\/\/preview.thenewsmarket.com\/Previews\/PWC\/DocumentAssets\/476830.pdf"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2797115.2797118"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/1060745.1060776"},{"key":"e_1_2_2_49_1","unstructured":"SAP SE. 2024. SAP Enterprise Architecture Framework. https:\/\/help.sap.com\/docs\/SAP_ENTERPRISE_ARCHITECTURE_FRAMEWORK Accessed: 2024-11-25."},{"key":"e_1_2_2_50_1","unstructured":"Maximilian Schreiner. 2023. GPT-4 architecture datasets costs and more leaked. The Decoder (online). https:\/\/the-decoder.com\/gpt-4-architecture-datasets-costs-and-more-leaked\/"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3661304.3661901"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.2200\/S01105ED1V01Y202105DSK020"},{"key":"e_1_2_2_53_1","volume-title":"Proceedings of the Workshop on Ontology Patterns co-located with the International Semantic Web Conference (WOP@ISWC). CEUR-WS.org, 97-108","author":"Sequeda Juan","year":"2012","unstructured":"Juan Sequeda, Freddy Priyatna, and Boris Villaz\u00f3n-Terrazas. 2012. Relational database to RDF mapping patterns. In Proceedings of the Workshop on Ontology Patterns co-located with the International Semantic Web Conference (WOP@ISWC). CEUR-WS.org, 97-108."},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0269888911000208"},{"key":"e_1_2_2_55_1","volume-title":"Proceedings of the ISWC Posters & Demonstrations Track co-located with the International Semantic Web Conference (ISWC) (CEUR Workshop Proceedings)","volume":"1486","author":"Juan","unstructured":"Juan F. Sequeda and Daniel P. Miranker. 2015. Ultrawrap Mapper: A Semi-Automatic Relational Database to RDF (RDB2RDF) Mapping Tool. In Proceedings of the ISWC Posters & Demonstrations Track co-located with the International Semantic Web Conference (ISWC) (CEUR Workshop Proceedings), Vol. 1486. http:\/\/ceur-ws.org\/Vol-1486\/paper_105.pdf"},{"key":"e_1_2_2_56_1","volume-title":"Proceedings of the Workshop on OWL: Experiences and Directions (OWLED). https:\/\/ceur-ws.org\/Vol-216\/submission_5.pdf","author":"Shen Guohua","year":"2006","unstructured":"Guohua Shen, Zhiqiu Huang, Xiaodong Zhu, and Xiaofei Zhao. 2006. Research on the Rules of Mapping from Relational Model to OWL. In Proceedings of the Workshop on OWL: Experiences and Directions (OWLED). https:\/\/ceur-ws.org\/Vol-216\/submission_5.pdf"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-41338-4_11"},{"key":"e_1_2_2_58_1","volume-title":"Proceedings of IEEE Workshop on Knowledge Acquisition from Distributed, Autonomous, Semantically Heterogeneous Data and Knowledge Sources.","author":"Tartir Samir","year":"2005","unstructured":"Samir Tartir, I. Budak Arpinar, Michael Moore, Amit P. Sheth, and Boanerges Aleman-Meza. 2005. OntoQA: Metric-Based Ontology Quality Analysis. In Proceedings of IEEE Workshop on Knowledge Acquisition from Distributed, Autonomous, Semantically Heterogeneous Data and Knowledge Sources."},{"key":"e_1_2_2_59_1","volume-title":"Mikel Ega\u00f1a Aranguren, and Jesualdo Tomas Fernandez Breis","author":"Calvo Mikel Val","year":"2024","unstructured":"Mikel Val Calvo, Mikel Ega\u00f1a Aranguren, and Jesualdo Tomas Fernandez Breis. 2024. OntoGenix. https:\/\/github.com\/tecnomod-um\/OntoGenix"},{"key":"e_1_2_2_60_1","volume-title":"Proceedings of the Semantic Web Challenge on Tabular Data to Knowledge Graph Matching co-located with the International Semantic Web Conference (SemTab@ISWC) (CEUR Workshop Proceedings)","volume":"2553","author":"Vandewiele Gilles","year":"2019","unstructured":"Gilles Vandewiele, Bram Steenwinckel, Filip De Turck, and Femke Ongenae. 2019. CVS2KG: Transforming Tabular Data into Semantic Knowledge. In Proceedings of the Semantic Web Challenge on Tabular Data to Knowledge Graph Matching co-located with the International Semantic Web Conference (SemTab@ISWC) (CEUR Workshop Proceedings), Vol. 2553. 33-40. https:\/\/ceur-ws.org\/Vol-2553\/paper5.pdf"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-34002-4_11"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-23620-4_8"},{"key":"e_1_2_2_63_1","volume-title":"OWL 2 Web Ontology Language Document Overview","author":"World Wide Web Consortium","unstructured":"World Wide Web Consortium. 2012. OWL 2 Web Ontology Language Document Overview (Second Edition). https:\/\/www.w3.org\/TR\/owl2-syntax\/."},{"key":"e_1_2_2_64_1","volume-title":"A Direct Mapping of Relational Data to RDF","author":"World Wide Web Consortium (W3C). 2012a.","year":"2025","unstructured":"World Wide Web Consortium (W3C). 2012a. A Direct Mapping of Relational Data to RDF. World Wide Web Consortium Recommendation. https:\/\/www.w3.org\/TR\/rdb-direct-mapping\/ Accessed: 2025-07-26."},{"key":"e_1_2_2_65_1","volume-title":"R2RML: RDB to RDF Mapping Language","author":"World Wide Web Consortium (W3C). 2012b.","year":"2024","unstructured":"World Wide Web Consortium (W3C). 2012b. R2RML: RDB to RDF Mapping Language. World Wide Web Consortium Recommendation. https:\/\/www.w3.org\/TR\/r2rml\/ Accessed: 2024-11-26."},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2018\/777"},{"key":"e_1_2_2_67_1","unstructured":"An Yang Anfeng Li Baosong Yang Beichen Zhang Binyuan Hui Bo Zheng Bowen Yu Chang Gao Chengen Huang Chenxu Lv and further authors. 2025. Qwen3 Technical Report. CoRR Vol. abs\/2505.09388 (2025). https:\/\/doi.org\/10.48550\/ARXIV.2505.09388 arXiv:2505.09388"},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.3233\/SW-170268"},{"key":"e_1_2_2_69_1","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR). https:\/\/openreview.net\/forum?id=92gvk82DE-","author":"Zhou Yongchao","year":"2023","unstructured":"Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2023. Large Language Models are Human-Level Prompt Engineers. In Proceedings of the International Conference on Learning Representations (ICLR). https:\/\/openreview.net\/forum?id=92gvk82DE-"}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3769770","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T04:55:02Z","timestamp":1781326502000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3769770"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,4]]},"references-count":69,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,12,4]]}},"alternative-id":["10.1145\/3769770"],"URL":"https:\/\/doi.org\/10.1145\/3769770","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,4]]}}}