{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T05:00:09Z","timestamp":1768280409823,"version":"3.49.0"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"11","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,7]]},"abstract":"<jats:p>\n            Training GNNs over large graphs faces a severe data processing bottleneck, involving both sampling and feature loading. To tackle this issue, we introduce F\n            <jats:sup>2<\/jats:sup>\n            CGT, a fast GNN training system incorporating feature compression. To avoid potential accuracy degradation, we propose a two-level, hybrid feature compression approach that applies different compression methods to various graph nodes. This differentiated choice strikes a balance between rounding errors, compression ratios, model accuracy loss, and preprocessing costs. Our theoretical analysis proves that this approach offers convergence and comparable model accuracy as the conventional training without feature compression. Additionally, we also co-design the on-GPU cache sub-system with compression-enabled training within F\n            <jats:sup>2<\/jats:sup>\n            CGT. The new cache sub-system, driven by a cost model, runs new cache policies to carefully choose graph nodes with high access frequencies, and well partitions the spare GPU memory for various types of graph data, for improving cache hit rates. Finally, extensive evaluation of F\n            <jats:sup>2<\/jats:sup>\n            CGT on two popular GNN models and four datasets, including three large public datasets, demonstrates that F\n            <jats:sup>2<\/jats:sup>\n            CGT achieves a compression ratio of up to 128 and provides GNN training speedups of 1.23-2.56\u00d7 and 3.58--71.46\u00d7 for single-machine and distributed training, respectively, with up to 32 GPUs and marginal accuracy loss.\n          <\/jats:p>","DOI":"10.14778\/3681954.3681968","type":"journal-article","created":{"date-parts":[[2024,8,30]],"date-time":"2024-08-30T16:23:36Z","timestamp":1725035016000},"page":"2854-2866","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Eliminating Data Processing Bottlenecks in GNN Training over Large Graphs via Two-level Feature Compression"],"prefix":"10.14778","volume":"17","author":[{"given":"Yuxin","family":"Ma","sequence":"first","affiliation":[{"name":"USTC"}]},{"given":"Ping","family":"Gong","sequence":"additional","affiliation":[{"name":"USTC"}]},{"given":"Tianming","family":"Wu","sequence":"additional","affiliation":[{"name":"USTC"}]},{"given":"Jiawei","family":"Yi","sequence":"additional","affiliation":[{"name":"USTC"}]},{"given":"Chengru","family":"Yang","sequence":"additional","affiliation":[{"name":"USTC"}]},{"given":"Cheng","family":"Li","sequence":"additional","affiliation":[{"name":"USTC and Institute of Artificial Intelligence, Hefei Comprehensive National Science Center"}]},{"given":"Qirong","family":"Peng","sequence":"additional","affiliation":[{"name":"OPPO"}]},{"given":"Guiming","family":"Xie","sequence":"additional","affiliation":[{"name":"OPPO"}]},{"given":"Yongcheng","family":"Bao","sequence":"additional","affiliation":[{"name":"OPPO"}]},{"given":"Haifeng","family":"Liu","sequence":"additional","affiliation":[{"name":"OPPO"}]},{"given":"Yinlong","family":"Xu","sequence":"additional","affiliation":[{"name":"USTC"}]}],"member":"320","published-online":{"date-parts":[[2024,8,30]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3477141"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1080\/00401706.1971.10488811"},{"key":"e_1_2_1_3_1","volume-title":"F2CGT open source code. https:\/\/github.com\/gpzlx1\/F2CGT\/. [Online","author":"Anonymous","year":"2024","unstructured":"Anonymous. 2024. F2CGT open source code. https:\/\/github.com\/gpzlx1\/F2CGT\/. [Online; accessed July-2024]."},{"key":"e_1_2_1_4_1","volume-title":"F2CGT supplemental material. https:\/\/github.com\/gpzlx1\/F2CGT-supplemental. [Online","author":"Anonymous","year":"2024","unstructured":"Anonymous. 2024. F2CGT supplemental material. https:\/\/github.com\/gpzlx1\/F2CGT-supplemental. [Online; accessed July-2024]."},{"key":"e_1_2_1_5_1","volume-title":"NVIDIA CUDA Unified Addressing. https:\/\/docs.nvidia.com\/cuda\/cuda-driver-api\/group___CUDA__UNIFIED.html. [Online","author":"NVIDIA Corporation","year":"2024","unstructured":"NVIDIA Corporation. 2023. NVIDIA CUDA Unified Addressing. https:\/\/docs.nvidia.com\/cuda\/cuda-driver-api\/group___CUDA__UNIFIED.html. [Online; accessed July-2024]."},{"key":"e_1_2_1_6_1","volume-title":"https:\/\/www.dgl.ai\/. [Online","author":"Homepage Team DGL.","year":"2024","unstructured":"Team DGL. 2023. DGL Homepage. https:\/\/www.dgl.ai\/. [Online; accessed July-2024]."},{"key":"e_1_2_1_7_1","first-page":"6733","article-title":"VQ-GNN: A universal framework to scale up graph neural networks using vector quantization","volume":"34","author":"Ding Mucong","year":"2021","unstructured":"Mucong Ding, Kezhi Kong, Jingling Li, Chen Zhu, John Dickerson, Furong Huang, and Tom Goldstein. 2021. VQ-GNN: A universal framework to scale up graph neural networks using vector quantization. Advances in Neural Information Processing Systems 34 (2021), 6733--6746.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467437"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICTAI50040.2020.00198"},{"key":"e_1_2_1_10_1","volume-title":"International Conference on Machine Learning. PMLR, 3294--3304","author":"Fey Matthias","year":"2021","unstructured":"Matthias Fey, Jan E Lenssen, Frank Weichert, and Jure Leskovec. 2021. Gnnautoscale: Scalable and expressive graph neural networks via historical embeddings. In International Conference on Machine Learning. PMLR, 3294--3304."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3568022"},{"key":"e_1_2_1_12_1","volume-title":"Convergence of adversarial training in overparametrized neural networks. Advances in Neural Information Processing Systems 32","author":"Gao Ruiqi","year":"2019","unstructured":"Ruiqi Gao, Tianle Cai, Haochuan Li, Cho-Jui Hsieh, Liwei Wang, and Jason D Lee. 2019. Convergence of adversarial training in overparametrized neural networks. Advances in Neural Information Processing Systems 32 (2019)."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3600006.3613168"},{"key":"e_1_2_1_14_1","volume-title":"Compressing deep convolutional networks using vector quantization. arXiv preprint arXiv:1412.6115","author":"Gong Yunchao","year":"2014","unstructured":"Yunchao Gong, Liu Liu, Ming Yang, and Lubomir Bourdev. 2014. Compressing deep convolutional networks using vector quantization. arXiv preprint arXiv:1412.6115 (2014)."},{"key":"e_1_2_1_15_1","volume-title":"Inductive representation learning on large graphs. Advances in neural information processing systems 30","author":"Hamilton Will","year":"2017","unstructured":"Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33709-3_54"},{"key":"e_1_2_1_17_1","volume-title":"OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs. arXiv preprint arXiv:2103.09430","author":"Hu Weihua","year":"2021","unstructured":"Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, and Jure Leskovec. 2021. OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs. arXiv preprint arXiv:2103.09430 (2021)."},{"key":"e_1_2_1_18_1","volume-title":"Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems 33","author":"Hu Weihua","year":"2020","unstructured":"Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems 33 (2020), 22118--22133."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447786.3456244"},{"key":"e_1_2_1_20_1","volume-title":"METIS: A software package for partitioning unstructured graphs, partitioning meshes, and computing fill-reducing orderings of sparse matrices.","author":"Karypis George","year":"1997","unstructured":"George Karypis and Vipin Kumar. 1997. METIS: A software package for partitioning unstructured graphs, partitioning meshes, and computing fill-reducing orderings of sparse matrices. (1997)."},{"key":"e_1_2_1_21_1","volume-title":"Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations.","author":"Thomas","unstructured":"Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2023.126441"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/3294771.3294804"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3419111.3421281"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346458"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2006.143"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102422"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/SC41405.2020.00060"},{"key":"e_1_2_1_29_1","volume-title":"PyTorch Geometric. https:\/\/pyg.org\/. [Online","author":"Team","year":"2024","unstructured":"Team PyG. 2023. PyTorch Geometric. https:\/\/pyg.org\/. [Online; accessed July-2024]."},{"key":"e_1_2_1_30_1","volume-title":"https:\/\/pytorch.org\/. [Online","author":"PyTorch Team","year":"2024","unstructured":"Team PyTorch. 2023. Pytorch Homepage. https:\/\/pytorch.org\/. [Online; accessed July-2024]."},{"key":"e_1_2_1_31_1","volume-title":"DeepRank-GNN: a graph neural network framework to learn patterns in protein-protein interfaces. Bioinformatics 39, 1","author":"R\u00e9au Manon","year":"2023","unstructured":"Manon R\u00e9au, Nicolas Renaud, Li C Xue, and Alexandre MJJ Bonvin. 2023. DeepRank-GNN: a graph neural network framework to learn patterns in protein-protein interfaces. Bioinformatics 39, 1 (2023), btac759."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2018.04.004"},{"key":"e_1_2_1_33_1","unstructured":"T. Konstantin Rusch Michael M. Bronstein and Siddhartha Mishra. 2023. A Survey on Oversmoothing in Graph Neural Networks. arXiv:2303.10993 [cs.LG]"},{"key":"e_1_2_1_34_1","unstructured":"Amazon Web Services. 2023. Amazon EC2 G4 Instances. https:\/\/aws.amazon.com\/ec2\/instance-types\/g4\/. accessed July-2024."},{"key":"e_1_2_1_35_1","volume-title":"Legion: Automatically Pushing the Envelope of Multi-GPU System for Billion-Scale GNN Training. In 2023 USENIX Annual Technical Conference (USENIX ATC 23)","author":"Sun Jie","year":"2023","unstructured":"Jie Sun, Li Su, Zuocheng Shi, Wenting Shen, Zeke Wang, Lei Wang, Jie Zhang, Yong Li, Wenyuan Yu, Jingren Zhou, and Fei Wu. 2023. Legion: Automatically Pushing the Envelope of Multi-GPU System for Billion-Scale GNN Training. In 2023 USENIX Annual Technical Conference (USENIX ATC 23). USENIX Association, Boston, MA, 165--179. https:\/\/www.usenix.org\/conference\/atc23\/presentation\/sun"},{"key":"e_1_2_1_36_1","volume-title":"Legion: Automatically Pushing the Envelope of Multi-GPU System for Billion-Scale GNN Training. In 2023 USENIX Annual Technical Conference (USENIX ATC 23)","author":"Sun Jie","year":"2023","unstructured":"Jie Sun, Li Su, Zuocheng Shi, Wenting Shen, Zeke Wang, Lei Wang, Jie Zhang, Yong Li, Wenyuan Yu, Jingren Zhou, and Fei Wu. 2023. Legion: Automatically Pushing the Envelope of Multi-GPU System for Billion-Scale GNN Training. In 2023 USENIX Annual Technical Conference (USENIX ATC 23). USENIX Association, Boston, MA, 165--179. https:\/\/www.usenix.org\/conference\/atc23\/presentation\/sun"},{"key":"e_1_2_1_37_1","volume-title":"Degree-quant: Quantization-aware training for graph neural networks. arXiv preprint arXiv:2008.05000","author":"Tailor Shyam A","year":"2020","unstructured":"Shyam A Tailor, Javier Fernandez-Marques, and Nicholas D Lane. 2020. Degree-quant: Quantization-aware training for graph neural networks. arXiv preprint arXiv:2008.05000 (2020)."},{"key":"e_1_2_1_38_1","unstructured":"SNAP Team. 2023. Friendster social network and ground-truth communities. https:\/\/snap.stanford.edu\/data\/com-Friendster.html. accessed July-2024."},{"key":"e_1_2_1_39_1","volume-title":"Graph attention networks. arXiv preprint arXiv:1710.10903","author":"Veli\u010dkovi\u0107 Petar","year":"2017","unstructured":"Petar Veli\u010dkovi\u0107, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)."},{"key":"e_1_2_1_40_1","volume-title":"Proceedings of Machine Learning and Systems 5","author":"Wan Borui","year":"2023","unstructured":"Borui Wan, Juntao Zhao, and Chuan Wu. 2023. Adaptive message quantization and parallelization for distributed full-graph gnn training. Proceedings of Machine Learning and Systems 5 (2023)."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00161"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1162\/qss_a_00021"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/19.492748"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3492321.3519557"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589311"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM53939.2023.10228911"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/IA351965.2020.00011"},{"key":"e_1_2_1_48_1","volume-title":"A2Q: Aggregation-Aware Quantization for Graph Neural Networks. arXiv preprint arXiv:2302.00193","author":"Zhu Zeyu","year":"2023","unstructured":"Zeyu Zhu, Fanrong Li, Zitao Mo, Qinghao Hu, Gang Li, Zejian Liu, Xiaoyao Liang, and Jian Cheng. 2023. A2Q: Aggregation-Aware Quantization for Graph Neural Networks. arXiv preprint arXiv:2302.00193 (2023)."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3681954.3681968","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T18:28:49Z","timestamp":1725474529000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3681954.3681968"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7]]},"references-count":48,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2024,7]]}},"alternative-id":["10.14778\/3681954.3681968"],"URL":"https:\/\/doi.org\/10.14778\/3681954.3681968","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,7]]},"assertion":[{"value":"2024-08-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}