{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T09:47:04Z","timestamp":1790934424984,"version":"4.1.0"},"reference-count":71,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,8]]},"abstract":"<jats:p>Based on the diversified application scenarios at Ant Group, we built the Ant Knowledge Graph Platform (AKGP). It has constructed numerous domain-specific knowledge graphs related to merchants, companies, accounts, products, and more. AKGP manages trillions of structured knowledge graphs, serving search, recommendation, risk control and other businesses. However, as the demand increasing for various workloads such as graph pattern matching, graph representation learning, and cross-domain knowledge reuse, the existing warehouse systems based on relational DBMS or graph databases are unable to meet the requirements. To address these issues, we propose KGFabric, an industrial-scale knowledge graph management system built on the distributed file system (DFS). KGFabric offers a nearline knowledge storage engine that utilizes a Semantic-enhanced Programmable Graph (SPG) model, which is compatible with the Labeled Property Graph (LPG) model. The data is persistently stored in DFS, such as HDFS, which leverages the POSIX file system API, making it suitable for deployment in multi-cloud environment at low cost. KGFabric provides a native graph-based and hybrid storage format that can serve as a shared backend for parallel graph computing systems, significantly accelerating the analysis of multi-workload. Additionally, KGFabric includes a graph fabric framework that minimizes data duplication and guarantees data security.<\/jats:p>\n          <jats:p>KGFabric is able to manage Peta-scale data and has supported graph fabric and analysis with over 100 billion relations at Ant Group. We conduct experiments on various datasets to evaluate the performance of KGFabric. Compared with popular relational DBMS and graph databases, the storage space for semantic relations is reduced by over 90%. The performance of graph fabric improves by 21\u00d7 in real-world workloads. In multi-hop semantic graph analysis, KGFabric enhances performance by 100\u00d7.<\/jats:p>","DOI":"10.14778\/3685800.3685810","type":"journal-article","created":{"date-parts":[[2024,11,8]],"date-time":"2024-11-08T17:25:21Z","timestamp":1731086721000},"page":"3841-3854","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["KGFabric: A Scalable Knowledge Graph Warehouse for Enterprise Data Interconnection"],"prefix":"10.14778","volume":"17","author":[{"given":"Peng","family":"Yi","sequence":"first","affiliation":[{"name":"Ant Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lei","family":"Liang","sequence":"additional","affiliation":[{"name":"Ant Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Da","family":"Zhang","sequence":"additional","affiliation":[{"name":"Ant Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yong","family":"Chen","sequence":"additional","affiliation":[{"name":"Ant Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinye","family":"Zhu","sequence":"additional","affiliation":[{"name":"Ant Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiangyu","family":"Liu","sequence":"additional","affiliation":[{"name":"Ant Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kun","family":"Tang","sequence":"additional","affiliation":[{"name":"Ant Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jialin","family":"Chen","sequence":"additional","affiliation":[{"name":"Ant Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Lin","sequence":"additional","affiliation":[{"name":"Ant Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leijie","family":"Qiu","sequence":"additional","affiliation":[{"name":"Ant Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jun","family":"Zhou","sequence":"additional","affiliation":[{"name":"Ant Group"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,11,8]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jnca.2021.103076"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.14778\/2757807.2757810"},{"key":"e_1_2_1_3_1","volume-title":"A survey of RDF stores & SPARQL engines for querying knowledge graphs. The VLDB Journal","author":"Ali Waqas","year":"2022","unstructured":"Waqas Ali, Muhammad Saleem, Bin Yao, Aidan Hogan, and Axel-Cyrille Ngonga Ngomo. 2022. A survey of RDF stores & SPARQL engines for querying knowledge graphs. The VLDB Journal (2022), 1--26."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589778"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457561"},{"key":"e_1_2_1_6_1","volume-title":"An introduction to graph data management. Graph Data Management: Fundamental Issues and Recent Developments","author":"Angles Renzo","year":"2018","unstructured":"Renzo Angles and Claudio Gutierrez. 2018. An introduction to graph data management. Graph Data Management: Fundamental Issues and Recent Developments (2018), 1--32."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2993117"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415560"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2915231"},{"key":"e_1_2_1_10_1","volume-title":"BitMat: A main memory RDF triple store. Tetherless World Constellation","author":"Atre Medha","year":"2009","unstructured":"Medha Atre, Jagannathan Srinivasan, and James A Hendler. 2009. BitMat: A main memory RDF triple store. Tetherless World Constellation, Rensselar Plytehcnic Institute, Troy NY (2009)."},{"key":"e_1_2_1_11_1","volume-title":"Amazon Neptune: Graph Data Management in the Cloud.. In ISWC (P&D\/Industry\/BlueSky).","author":"Bebee Bradley R","year":"2018","unstructured":"Bradley R Bebee, Daniel Choi, Ankit Gupta, Andi Gutmans, Ankesh Khandelwal, Yigit Kiran, Sainath Mallidi, Bruce McGaughy, Mike Personick, Karthik Rajan, et al. 2018. Amazon Neptune: Graph Data Management in the Cloud.. In ISWC (P&D\/Industry\/BlueSky)."},{"key":"e_1_2_1_12_1","volume-title":"Retrieved","year":"2023","unstructured":"BenchCases 2023. BenchCases. Retrieved November 24, 2023 from http:\/\/github.com\/FessGo\/knowledge-graph-warehouse-bench"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.5555\/2019470.2019472"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3439724"},{"key":"e_1_2_1_15_1","volume-title":"Better bitmap performance with roaring bitmaps. Software: practice and experience 46, 5","author":"Chambi Samy","year":"2016","unstructured":"Samy Chambi, Daniel Lemire, Owen Kaser, and Robert Godin. 2016. Better bitmap performance with roaring bitmaps. Software: practice and experience 46, 5 (2016), 709--719."},{"key":"e_1_2_1_16_1","volume-title":"Retrieved","author":"Concept","year":"2023","unstructured":"Concept 2023. Concept. Retrieved November 11, 2023 from http:\/\/en.wikipedia.org\/wiki\/Concept"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3526057"},{"key":"e_1_2_1_18_1","volume-title":"Tsunami: A learned multi-dimensional index for correlated data and skewed workloads. arXiv preprint arXiv:2006.13282","author":"Ding Jialin","year":"2020","unstructured":"Jialin Ding, Vikram Nathan, Mohammad Alizadeh, and Tim Kraska. 2020. Tsunami: A learned multi-dimensional index for correlated data and skewed workloads. arXiv preprint arXiv:2006.13282 (2020)."},{"key":"e_1_2_1_19_1","first-page":"3","article-title":"Optimizing Space Amplification in RocksDB","volume":"3","author":"Dong Siying","year":"2017","unstructured":"Siying Dong, Mark Callaghan, Leonidas Galanis, Dhruba Borthakur, Tony Savor, and Michael Strum. 2017. Optimizing Space Amplification in RocksDB.. In CIDR, Vol. 3. 3.","journal-title":"CIDR"},{"key":"e_1_2_1_20_1","first-page":"3","article-title":"Virtuoso, a Hybrid RDBMS\/Graph Column Store","volume":"35","author":"Erling Orri","year":"2012","unstructured":"Orri Erling. 2012. Virtuoso, a Hybrid RDBMS\/Graph Column Store. IEEE Data Eng. Bull. 35, 1 (2012), 3--8.","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33013558"},{"key":"e_1_2_1_22_1","volume-title":"Apache Flink --- Stateful Computations over Data Streams \u2026 Retrieved","author":"Flink","year":"2023","unstructured":"Flink 2023. Apache Flink --- Stateful Computations over Data Streams \u2026 Retrieved October 10, 2023 from http:\/\/flink.apache.org\/"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3575637.3575644"},{"key":"e_1_2_1_24_1","volume-title":"Retrieved","author":"Gartner","year":"2023","unstructured":"Gartner: Data Fabric 2023. Using Data Fabric Architecture to Modernize Data Integration. Retrieved October 25, 2023 from https:\/\/www.gartner.com\/smarterwithgartner\/data-fabric-architecture-is-key-to-modernizing-data-management-and-integration"},{"key":"e_1_2_1_25_1","volume-title":"10th USENIX symposium on operating systems design and implementation (OSDI 12)","author":"Gonzalez Joseph E","year":"2012","unstructured":"Joseph E Gonzalez, Yucheng Low, Haijie Gu, Danny Bickson, and Carlos Guestrin. 2012. {PowerGraph}: Distributed {Graph-Parallel} computation on natural graphs. In 10th USENIX symposium on operating systems design and implementation (OSDI 12). 17--30."},{"key":"e_1_2_1_26_1","volume-title":"11th USENIX symposium on operating systems design and implementation (OSDI 14)","author":"Gonzalez Joseph E","year":"2014","unstructured":"Joseph E Gonzalez, Reynold S Xin, Ankur Dave, Daniel Crankshaw, Michael J Franklin, and Ion Stoica. 2014. {GraphX}: Graph processing in a distributed dataflow framework. In 11th USENIX symposium on operating systems design and implementation (OSDI 14). 599--613."},{"key":"e_1_2_1_27_1","volume-title":"Retrieved","author":"Graph Google Knowledge","year":"2024","unstructured":"Google Knowledge Graph 2024. Google Knowledge Graph. Retrieved February 20, 2024 from http:\/\/en.wikipedia.org\/wiki\/Google_Knowledge_Graph"},{"key":"e_1_2_1_28_1","unstructured":"Google S2 2023. S2 Geometry | S2Geometry. Retrieved November 7 2023 from http:\/\/s2geometry.io\/"},{"key":"e_1_2_1_29_1","volume-title":"Retrieved","author":"GQL","year":"2023","unstructured":"GQL 2023. Graph Query Language GQL. Retrieved October 30, 2023 from http:\/\/www.gqlstandards.org"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2402.14710"},{"key":"e_1_2_1_31_1","volume-title":"Columnar storage and list-based processing for graph database management systems. arXiv preprint arXiv:2103.02284","author":"Gupta Pranjal","year":"2021","unstructured":"Pranjal Gupta, Amine Mhedhbi, and Semih Salihoglu. 2021. Columnar storage and list-based processing for graph database management systems. arXiv preprint arXiv:2103.02284 (2021)."},{"key":"e_1_2_1_32_1","volume-title":"Retrieved","author":"Hadoop","year":"2023","unstructured":"Hadoop 2023. Apache Hadoop. Retrieved October 25, 2023 from http:\/\/hadoop.apache.org"},{"key":"e_1_2_1_33_1","unstructured":"Olaf Hartig. 2017. RDF* and SPARQL*: An Alternative Approach to Annotate Statements in RDF.. In ISWC (Posters Demos & Industry Tracks)."},{"key":"e_1_2_1_34_1","volume-title":"Foundations of an alternative approach to reification in RDF. arXiv preprint arXiv:1406.3399","author":"Hartig Olaf","year":"2014","unstructured":"Olaf Hartig and Bryan Thompson. 2014. Foundations of an alternative approach to reification in RDF. arXiv preprint arXiv:1406.3399 (2014)."},{"key":"e_1_2_1_35_1","volume-title":"Retrieved","author":"Hive","year":"2023","unstructured":"Hive 2023. Apache Hive. Retrieved October 11, 2023 from http:\/\/hive.apache.org"},{"key":"e_1_2_1_36_1","volume-title":"Retrieved","author":"Index JanusGraph","year":"2023","unstructured":"JanusGraph Index 2023. Indexing for Better Performance. Retrieved October 11, 2023 from http:\/\/docs.janusgraph.org\/schema\/index-management\/index-performance"},{"key":"e_1_2_1_37_1","volume-title":"Gradoop: Scalable graph data management and analytics with hadoop. arXiv preprint arXiv:1506.00548","author":"Junghanns Martin","year":"2015","unstructured":"Martin Junghanns, Andr\u00e9 Petermann, Kevin G\u00f3mez, and Erhard Rahm. 2015. Gradoop: Scalable graph data management and analytics with hadoop. arXiv preprint arXiv:1506.00548 (2015)."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3442381.3450010"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.14778\/3554821.3554824"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-019-00555-y"},{"key":"e_1_2_1_41_1","volume-title":"Differentiating concepts and instances for knowledge graph embedding. arXiv preprint arXiv:1811.04588","author":"Lv Xin","year":"2018","unstructured":"Xin Lv, Lei Hou, Juanzi Li, and Zhiyuan Liu. 2018. Differentiating concepts and instances for knowledge graph embedding. arXiv preprint arXiv:1811.04588 (2018)."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2015.7113298"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807167.1807184"},{"key":"e_1_2_1_44_1","volume-title":"The Rise of the Knowledge Graph","author":"Martin Sean","unstructured":"Sean Martin, Ben Szekely, and Dean Allemang. 2021. The Rise of the Knowledge Graph. O'Reilly Media, Incorporated."},{"key":"e_1_2_1_45_1","volume-title":"Retrieved","year":"2023","unstructured":"MaxCompute 2023. MaxCompute: Conduct Petabyte-Scale Data Warehousing. Retrieved October 11, 2023 from https:\/\/www.alibabacloud.com\/product\/maxcompute"},{"key":"e_1_2_1_46_1","volume-title":"Optimizing subgraph queries by combining binary and worst-case optimal joins. arXiv preprint arXiv:1903.02076","author":"Mhedhbi Amine","year":"2019","unstructured":"Amine Mhedhbi and Semih Salihoglu. 2019. Optimizing subgraph queries by combining binary and worst-case optimal joins. arXiv preprint arXiv:1903.02076 (2019)."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.14778\/3352063.3352116"},{"key":"e_1_2_1_48_1","volume-title":"Retrieved","year":"2023","unstructured":"Neo4j 2023. Neo4j Graph Database Analytics | Graph Database. Retrieved October 20, 2023 from http:\/\/neo4j.com"},{"key":"e_1_2_1_49_1","volume-title":"Retrieved","year":"2023","unstructured":"Neo4j Fabric 2023. Sharding Graph Data with Neo4j Fabric -Developer Guides. Retrieved October 25, 2023 from http:\/\/neo4j.com\/developer\/neo4j-fabric-sharding\/"},{"key":"e_1_2_1_50_1","volume-title":"Retrieved","author":"Nepture","year":"2023","unstructured":"Nepture 2023. Amazon Neptune - Fully Managed Graph Database - AWS. Retrieved October 11, 2023 from http:\/\/aws.amazon.com\/neptune"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-009-0165-y"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3329781.3332266"},{"key":"e_1_2_1_53_1","volume-title":"Retrieved","author":"Ontotext","year":"2023","unstructured":"Ontotext GraphDB 2023. Ontotext. Retrieved October 11, 2023 from http:\/\/www.ontotext.com\/products\/graphdb"},{"key":"e_1_2_1_54_1","unstructured":"OpenSPG 2023. Retrieved October 30 2023 from http:\/\/github.com\/OpenSPG\/openspg"},{"key":"e_1_2_1_55_1","unstructured":"Alisdair Owens Andy Seaborne Nick Gibbins et al. 2008. Clustered TDB: A clustered triple store for Jena. (2008)."},{"key":"e_1_2_1_56_1","volume-title":"Unifying Large Language Models and Knowledge Graphs: A Roadmap. arXiv preprint arXiv:2306.08302","author":"Pan Shirui","year":"2023","unstructured":"Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. 2023. Unifying Large Language Models and Knowledge Graphs: A Roadmap. arXiv preprint arXiv:2306.08302 (2023)."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589771"},{"key":"e_1_2_1_58_1","unstructured":"Shipeng Qi Heng Lin Zhihui Guo G\u00e1bor Sz\u00e1rnyas Bing Tong Yan Zhou Bin Yang Jiansong Zhang Zheng Wang Youren Shen et al. 2023. The LDBC Financial Benchmark. arXiv preprint arXiv:2306.15975 (2023)."},{"key":"e_1_2_1_59_1","volume-title":"Retrieved","year":"2023","unstructured":"QLExpress 2023. alibaba\/QLExpress. Retrieved October 11, 2023 from http:\/\/github.com\/alibaba\/QLExpress"},{"key":"e_1_2_1_60_1","volume-title":"Graph databases: new opportunities for connected data. \"O'Reilly Media","author":"Robinson Ian","unstructured":"Ian Robinson, Jim Webber, and Emil Eifrem. 2015. Graph databases: new opportunities for connected data. \"O'Reilly Media, Inc.\"."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/2815072.2815073"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2467799"},{"key":"e_1_2_1_63_1","volume-title":"Retrieved","year":"2023","unstructured":"SOFAStack 2023. Scalable Open Financial Architecture Stack-Alibaba Cloud. Retrieved October 25, 2023 from http:\/\/www.alibabacloud.com\/product\/sofastack"},{"key":"e_1_2_1_64_1","volume-title":"Retrieved","author":"Stardog","year":"2023","unstructured":"Stardog 2023. Stardog: The Enterprise Knowledge Graph Platform. Retrieved October 11, 2023 from http:\/\/stardog.com"},{"key":"e_1_2_1_65_1","doi-asserted-by":"crossref","unstructured":"Patrick St\u00fcnkel Ole von Bargen Adrian Rutle and Yngve Lamo. 2020. GraphQL federation: A model-based approach. (2020).","DOI":"10.5381\/jot.2020.19.2.a18"},{"key":"e_1_2_1_66_1","volume-title":"Retrieved","author":"Tugraph","year":"2023","unstructured":"Tugraph 2023. TuGraph the high-performance graph database. Retrieved October 11, 2023 from http:\/\/tugraph.antgroup.com"},{"key":"e_1_2_1_67_1","volume-title":"Retrieved","year":"2023","unstructured":"TypeQL 2023. vaticle\/typeql - the polymorphic query language of TypeDB. Retrieved October 20, 2023 from https:\/\/github.com\/vaticle\/typeql"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380246"},{"key":"e_1_2_1_69_1","volume-title":"Nebula Graph: An open source distributed graph database. arXiv preprint arXiv:2206.07278","author":"Wu Min","year":"2022","unstructured":"Min Wu, Xinglu Yi, Hui Yu, Yu Liu, and Yujue Wang. 2022. Nebula Graph: An open source distributed graph database. arXiv preprint arXiv:2206.07278 (2022)."},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536349.2536352"},{"key":"e_1_2_1_71_1","volume-title":"Livegraph: A transactional graph storage system with purely sequential adjacency list scans. arXiv preprint arXiv:1910.05773","author":"Zhu Xiaowei","year":"2019","unstructured":"Xiaowei Zhu, Guanyu Feng, Marco Serafini, Xiaosong Ma, Jiping Yu, Lei Xie, Ashraf Aboulnaga, and Wenguang Chen. 2019. Livegraph: A transactional graph storage system with purely sequential adjacency list scans. arXiv preprint arXiv:1910.05773 (2019)."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3685800.3685810","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,31]],"date-time":"2024-12-31T05:35:02Z","timestamp":1735623302000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3685800.3685810"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8]]},"references-count":71,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2024,8]]}},"alternative-id":["10.14778\/3685800.3685810"],"URL":"https:\/\/doi.org\/10.14778\/3685800.3685810","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,8]]},"assertion":[{"value":"2024-11-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}