{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T23:22:58Z","timestamp":1771024978058,"version":"3.50.1"},"reference-count":77,"publisher":"Association for Computing Machinery (ACM)","issue":"3","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,11]]},"abstract":"<jats:p>Data lakes, increasingly adopted for their ability to store and analyze diverse types of data, commonly use columnar storage formats like Parquet and ORC for handling relational tables. However, these traditional setups fall short when it comes to efficiently managing graph data, particularly those conforming to the Labeled Property Graph (LPG) model. To address this gap, this paper introduces GraphAr, a specialized storage scheme designed to enhance existing data lakes for efficient graph data management. Leveraging the strengths of Parquet, GraphAr captures LPG semantics precisely and facilitates graph-specific operations such as neighbor retrieval and label filtering. Through innovative data organization, encoding, and decoding techniques, GraphAr dramatically improves performance. Our evaluations reveal that GraphAr outperforms conventional Parquet and Acero-based methods, achieving an average speedup of 4452\u00d7 for neighbor retrieval, 14.8\u00d7 for label filtering, and 29.5\u00d7 for end-to-end workloads. These findings highlight GraphAr's potential to extend the utility of data lakes by enabling efficient graph data management.<\/jats:p>","DOI":"10.14778\/3712221.3712223","type":"journal-article","created":{"date-parts":[[2025,4,7]],"date-time":"2025-04-07T18:03:04Z","timestamp":1744048984000},"page":"530-543","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["GraphAr: An Efficient Storage Scheme for Graph Data in Data Lakes"],"prefix":"10.14778","volume":"18","author":[{"given":"Xue","family":"Li","sequence":"first","affiliation":[{"name":"Alibaba Group"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weibin","family":"Zeng","sequence":"additional","affiliation":[{"name":"Alibaba Group"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhibin","family":"Wang","sequence":"additional","affiliation":[{"name":"Nanjing University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Diwen","family":"Zhu","sequence":"additional","affiliation":[{"name":"Alibaba Group"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jingbo","family":"Xu","sequence":"additional","affiliation":[{"name":"Alibaba Group"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenyuan","family":"Yu","sequence":"additional","affiliation":[{"name":"Alibaba Group"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jingren","family":"Zhou","sequence":"additional","affiliation":[{"name":"Alibaba Group"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,4,7]]},"reference":[{"key":"e_1_2_1_1_1","first-page":"2023","article-title":"Information technology - Database languages - SQL","volume":"9075","year":"2023","unstructured":"2023. Information technology - Database languages - SQL. ISO\/IEC 9075:2023.","journal-title":"ISO\/IEC"},{"key":"e_1_2_1_2_1","unstructured":"2024. Acero: A C++ streaming execution engine. https:\/\/arrow.apache.org\/docs\/cpp\/streaming_execution.html. Accessed on 2024-12-17."},{"key":"e_1_2_1_3_1","unstructured":"2024. Apache Arrow. https:\/\/arrow.apache.org\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_4_1","unstructured":"2024. Apache AVRO. https:\/\/avro.apache.org\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_5_1","unstructured":"2024. Apache Gobblin. https:\/\/gobblin.apache.org\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_6_1","unstructured":"2024. Apache ORC File Format. https:\/\/orc.apache.org\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_7_1","unstructured":"2024. Apache Parquet File Format. https:\/\/parquet.apache.org\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_8_1","unstructured":"2024. Apache Pinot. https:\/\/pinot.apache.org\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_9_1","unstructured":"2024. Apache TinkerPop. https:\/\/tinkerpop.apache.org\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_10_1","unstructured":"2024. AWS S3 Select. https:\/\/aws.amazon.com\/cn\/blogs\/aws\/s3-glacier-select\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_11_1","unstructured":"2024. Cypher Query language. https:\/\/neo4j.com\/developer\/cypher\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_12_1","unstructured":"2024. Export Neo4j Data to CSV. https:\/\/neo4j.com\/labs\/apoc\/4.4\/export\/csv\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_13_1","unstructured":"2024. GEXF File Format. https:\/\/gexf.net\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_14_1","unstructured":"2024. Graph 500 Benchmarks. https:\/\/graph500.org\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_15_1","unstructured":"2024. Intel(R) Threading Building Blocks. https:\/\/github.com\/wjakob\/tbb. Accessed on 2024-12-17."},{"key":"e_1_2_1_16_1","unstructured":"2024. JanusGraph: an open-source distributed graph database. https:\/\/janusgraph.org\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_17_1","unstructured":"2024. LDBC SNB Business Intelligence (BI) workload implementations. https:\/\/github.com\/ldbc\/ldbc_snb_bi. Accessed on 2024-12-17."},{"key":"e_1_2_1_18_1","unstructured":"2024. LDBC SNB Interactive workload implementations. https:\/\/github.com\/ldbc\/ldbc_snb_interactive_impls. Accessed on 2024-12-17."},{"key":"e_1_2_1_19_1","unstructured":"2024. Neo4j Graph Database and Analytics. https:\/\/neo4j.com\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_20_1","unstructured":"2024. Neo4j Graph Examples. https:\/\/github.com\/neo4j-graph-examples. Accessed on 2024-12-17."},{"key":"e_1_2_1_21_1","unstructured":"2024. Neo4j Spark Connector. https:\/\/neo4j.com\/docs\/spark\/current\/reading\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_22_1","unstructured":"2024. PuppyGraph: A Cloud-Native Graph Data Lakehouse. https:\/\/puppygraph.com\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_23_1","unstructured":"2024. RDF Formats. https:\/\/www.w3.org\/TR\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_24_1","unstructured":"2024. The HDF5 Library and File Format. https:\/\/www.hdfgroup.org\/solutions\/hdf5\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589778"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/IVMEM.2019.00007"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415560"},{"key":"e_1_2_1_28_1","volume-title":"LDAC2020---8th Linked Data in Architecture and Construction Workshop. 23--36","author":"Baken Nico","year":"2020","unstructured":"Nico Baken. 2020. Linked data for smart homes: Comparing RDF and labeled property graphs. In LDAC2020---8th Linked Data in Architecture and Construction Workshop. 23--36."},{"key":"e_1_2_1_29_1","volume-title":"International Symposium on Graph Drawing. Springer, 477--478","author":"Batagelj Vladimir","year":"2001","unstructured":"Vladimir Batagelj and Andrej Mrvar. 2001. Pajek---analysis and visualization of large networks. In International Symposium on Graph Drawing. Springer, 477--478."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3526054"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/988672.988752"},{"key":"e_1_2_1_32_1","volume-title":"GraphML Progress Report Structural Layer Proposal","author":"Brandes Ulrik","unstructured":"Ulrik Brandes, Markus Eiglsperger, Ivan Herman, Michael Himsolt, and M. Scott Marshall. 2002. GraphML Progress Report Structural Layer Proposal. In Graph Drawing, Petra Mutzel, Michael J\u00fcnger, and Sebastian Leipert (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 501--512."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2903741"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2049662.2049663"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3526057"},{"key":"e_1_2_1_36_1","volume-title":"Lee","author":"Deutsch Alin","year":"2019","unstructured":"Alin Deutsch, Yu Xu, Mingxi Wu, and Victor E. Lee. 2019. TigerGraph: A Native MPP Graph Database. CoRR abs\/1901.08248 (2019). arXiv:1901.08248 http:\/\/arxiv.org\/abs\/1901.08248"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2742786"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476311.3476369"},{"key":"e_1_2_1_39_1","unstructured":"Xiyang Feng Guodong Jin Ziyi Chen Chang Liu and Semih Saliho\u011flu. 2022. K\u00f9zu Database Management System Source Code. https:\/\/github.com\/kuzudb\/kuzu. Accessed on 2024-12-17."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732977.2733002"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.4230\/LIPIcs.ICDT.2023.1"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3190657"},{"key":"e_1_2_1_43_1","volume-title":"Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation","author":"Gonzalez Joseph E.","year":"2014","unstructured":"Joseph E. Gonzalez, Reynold S. Xin, Ankur Dave, Daniel Crankshaw, Michael J. Franklin, and Ion Stoica. 2014. GraphX: graph processing in a distributed dataflow framework. In Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation (Broomfield, CO) (OSDI'14). USENIX Association, USA, 599--613."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476249.3476297"},{"key":"e_1_2_1_45_1","volume-title":"GML: A portable Graph File Format. https:\/\/api.semanticscholar.org\/CorpusID:5806388. Accessed on 2024-12-17.","author":"Himsolt Michael","year":"2010","unstructured":"Michael Himsolt. 2010. GML: A portable Graph File Format. https:\/\/api.semanticscholar.org\/CorpusID:5806388. Accessed on 2024-12-17."},{"key":"e_1_2_1_46_1","unstructured":"Weihua Hu Matthias Fey Marinka Zitnik Yuxiao Dong Hongyu Ren Bowen Liu Michele Catasta and Jure Leskovec. 2021. Open Graph Benchmark: Datasets for Machine Learning on Graphs. (2021). arXiv:2005.00687 [cs.LG] https:\/\/arxiv.org\/abs\/2005.00687"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.5523"},{"key":"e_1_2_1_48_1","volume-title":"ITM web of conferences","author":"Khine Pwint Phyu","unstructured":"Pwint Phyu Khine and Zhao Shun Wang. 2018. Data lake: a new ideology in big data era. In ITM web of conferences, Vol. 17. EDP Sciences, 03025."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487788.2488173"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1002\/spe.2203"},{"key":"e_1_2_1_51_1","volume-title":"SNAP Datasets: Stanford Large Network Dataset Collection","author":"Leskovec Jure","unstructured":"Jure Leskovec and Andrej Krevl. 2014. SNAP Datasets: Stanford Large Network Dataset Collection. http:\/\/snap.stanford.edu\/data. Accessed: 2024-12-17."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589323"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-30284-8_36"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.14778\/3554821.3554894"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2016.07.439"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.14778\/3352063.3352116"},{"key":"e_1_2_1_57_1","unstructured":"Max Neunh\u00f6ffer. 2024. Fishing for graphs in a Hadoop data lake. https:\/\/www.oreilly.com\/content\/fishing-for-graphs-in-a-hadoop-data-lake\/. Accessed on 2024-12-17."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/2764947.2764948"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2747645"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3329785.3329928"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-020-00621-w"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3320212"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3056100"},{"key":"e_1_2_1_64_1","volume-title":"New Trends in Databases and Information Systems: ADBIS 2019 Short Papers, Workshops BBIGAP, QAUCA, SemBDM, SIMPDA, M2P, MADEISD","author":"Ravat Franck","year":"2019","unstructured":"Franck Ravat and Yan Zhao. 2019. Metadata management for data lakes. In New Trends in Databases and Information Systems: ADBIS 2019 Short Papers, Workshops BBIGAP, QAUCA, SemBDM, SIMPDA, M2P, MADEISD, and Doctoral Consortium, Bled, Slovenia, September 8--11, 2019, Proceedings 23. Springer, 37--44."},{"key":"e_1_2_1_65_1","volume-title":"The Gremlin Graph Traversal Machine and Language. CoRR abs\/1508.03843","author":"Rodriguez Marko A.","year":"2015","unstructured":"Marko A. Rodriguez. 2015. The Gremlin Graph Traversal Machine and Language. CoRR abs\/1508.03843 (2015). arXiv:1508.03843 http:\/\/arxiv.org\/abs\/1508.03843"},{"key":"e_1_2_1_66_1","volume-title":"Ahmed","author":"Rossi Ryan A.","year":"2015","unstructured":"Ryan A. Rossi and Nesreen K. Ahmed. 2015. The Network Data Repository with Interactive Graph Analytics and Visualization. In AAAI. https:\/\/networkrepository.com"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.14778\/3574245.3574270"},{"key":"e_1_2_1_68_1","unstructured":"Johan Ugander Brian Karrer Lars Backstrom and Cameron Marlow. 2011. The Anatomy of the Facebook Social Graph. (2011). arXiv:1111.4503 [cs.SI] https:\/\/arxiv.org\/abs\/1111.4503"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/2960414.2960421"},{"key":"e_1_2_1_70_1","volume-title":"Vectorizing Database Column Scans with Complex Predicates. In International Workshop on Accelerating Data Management Systems Using Modern Processor and Storage Architectures (ADMS 2013","author":"Willhalm Thomas","year":"2013","unstructured":"Thomas Willhalm, Ismail Oukid, Ingo M\u00fcller, and Franz Faerber. 2013. Vectorizing Database Column Scans with Complex Predicates. In International Workshop on Accelerating Data Management Systems Using Modern Processor and Storage Architectures (ADMS 2013), Riva del Garda, Trento, I, August 26, 2013. 1--12."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687627.1687671"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476249.3476265"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE48307.2020.00174"},{"key":"e_1_2_1_74_1","volume-title":"11th Conference on Innovative Data Systems Research, CIDR 2021, Virtual Event, January 11-15, 2021, Online Proceedings. www.cidrdb.org. http:\/\/cidrdb.org\/cidr2021\/papers\/cidr2021_paper17","author":"Zaharia Matei","year":"2021","unstructured":"Matei Zaharia, Ali Ghodsi 0002, Reynold Xin, and Michael Armbrust. 2021. Lake-house: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics. In 11th Conference on Innovative Data Systems Research, CIDR 2021, Virtual Event, January 11-15, 2021, Online Proceedings. www.cidrdb.org. http:\/\/cidrdb.org\/cidr2021\/papers\/cidr2021_paper17.pdf"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/2934664"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3035943"},{"key":"e_1_2_1_77_1","volume-title":"Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation","author":"Zhu Xiaowei","year":"2016","unstructured":"Xiaowei Zhu, Wenguang Chen, Weimin Zheng, and Xiaosong Ma. 2016. Gemini: a computation-centric distributed graph processing system. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (Savannah, GA, USA) (OSDI'16). USENIX Association, USA, 301--316."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3712221.3712223","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,7]],"date-time":"2025-04-07T18:13:22Z","timestamp":1744049602000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3712221.3712223"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11]]},"references-count":77,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,11]]}},"alternative-id":["10.14778\/3712221.3712223"],"URL":"https:\/\/doi.org\/10.14778\/3712221.3712223","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,11]]},"assertion":[{"value":"2025-04-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}