{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,30]],"date-time":"2025-08-30T00:06:06Z","timestamp":1756512366195,"version":"3.44.0"},"reference-count":84,"publisher":"Association for Computing Machinery (ACM)","issue":"6","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2025,2]]},"abstract":"<jats:p>Graph neural networks (GNNs) have emerged as a promising method for learning from graph data, but large-scale GNN training requires extensive memory and computation resources. To address this, researchers have proposed using multi-GPU processing, which partitions graph data across GPUs for parallel training. However, vertex dependencies in multi-GPU GNN training lead to significant neighbor replications across GPUs, increasing memory consumption. The substantial intermediate data generated during training further exacerbates this issue. Neighbor replication and intermediate data constitute the primary memory consumption in GNN training (i.e., typically accounting for over 80%). In this work, we propose GNN task parallelism for multi-GPU GNN training, which reduces neighbor replication by partitioning training tasks in each layer across different GPUs rather than partitioning the graph structure. This approach only partitions the graph data within individual GPUs, reducing the memory requirements of single tasks while overlapping subgraph computation across different GPUs. Shared neighbor embeddings among different subgraphs can be efficiently reused within a single GPU. Additionally, we employ a task-decoupled GNN training framework, which decouples different training tasks to manage their associated intermediate data independently and release it as early as possible to reduce memory usage. By integrating these techniques, we propose a multi-GPU GNN training system, NeutronTask. Experimental results on a 4\u00d7A5000 GPU server show that NeutronTask effectively supports billion-scale full-graph GNN training. For small graphs where the training data fits into the GPUs, NeutronTask achieves 1.27\u00d7 - 5.47\u00d7 speedup compared to state-of-the-art GNN systems including NeutronStar and Sancus.<\/jats:p>","DOI":"10.14778\/3725688.3725700","type":"journal-article","created":{"date-parts":[[2025,8,29]],"date-time":"2025-08-29T14:19:21Z","timestamp":1756477161000},"page":"1705-1719","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["NeutronTask: Scalable and Efficient Multi-GPU GNN Training with Task Parallelism"],"prefix":"10.14778","volume":"18","author":[{"given":"Zhenbo","family":"Fu","sequence":"first","affiliation":[{"name":"Northeastern Univ, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xin","family":"Ai","sequence":"additional","affiliation":[{"name":"Northeastern Univ, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qiange","family":"Wang","sequence":"additional","affiliation":[{"name":"National University of Singapore, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yanfeng","family":"Zhang","sequence":"additional","affiliation":[{"name":"Northeastern Univ, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shizhan","family":"Lu","sequence":"additional","affiliation":[{"name":"Northeastern Univ, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chaoyi","family":"Chen","sequence":"additional","affiliation":[{"name":"Northeastern Univ, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chunyu","family":"Cao","sequence":"additional","affiliation":[{"name":"Northeastern Univ, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hao","family":"Yuan","sequence":"additional","affiliation":[{"name":"Northeastern Univ, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhewei","family":"Wei","sequence":"additional","affiliation":[{"name":"Renmin University of China, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu","family":"Gu","sequence":"additional","affiliation":[{"name":"Northeastern Univ, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yingyou","family":"Wen","sequence":"additional","affiliation":[{"name":"Neusoft AI, Research, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ge","family":"Yu","sequence":"additional","affiliation":[{"name":"Northeastern Univ, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,8,29]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.14778\/3659437.3659453"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.14778\/3705829.3705837"},{"key":"e_1_2_1_3_1","volume-title":"Relations and Physics. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016","author":"Battaglia Peter W.","year":"2016","unstructured":"Peter W. Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, and Koray Kavukcuoglu. 2016. Interaction Networks for Learning about Objects, Relations and Physics. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5\u201310, 2016, Barcelona, Spain. 4502\u20134510."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/988672.988752"},{"key":"e_1_2_1_5_1","volume-title":"The Tenth International Conference on Learning Representations, ICLR 2022","author":"Brody Shaked","year":"2022","unstructured":"Shaked Brody, Uri Alon, and Eran Yahav. 2022. How Attentive are Graph Attention Networks?. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25\u201329, 2022."},{"key":"e_1_2_1_6_1","doi-asserted-by":"crossref","first-page":"2763","DOI":"10.1007\/s10489-021-02587-w","article-title":"Spatial-temporal graph neural network for traffic forecasting: An overview and open research issues","volume":"52","author":"Nam Bui Khac-Hoai","year":"2022","unstructured":"Khac-Hoai Nam Bui, Jiho Cho, and Hongsuk Yi. 2022. Spatial-temporal graph neural network for traffic forecasting: An overview and open research issues. Appl. Intell. 52, 3 (2022), 2763\u20132774.","journal-title":"Appl. Intell."},{"key":"e_1_2_1_7_1","volume-title":"A note on over-smoothing for graph neural networks. arXiv preprint arXiv:2006.13318","author":"Cai Chen","year":"2020","unstructured":"Chen Cai and Yusu Wang. 2020. A note on over-smoothing for graph neural networks. arXiv preprint arXiv:2006.13318 (2020)."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447786.3456233"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the 28th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming, PPoPP 2023","author":"Cai Zhenkun","year":"2023","unstructured":"Zhenkun Cai, Qihui Zhou, Xiao Yan, Da Zheng, Xiang Song, Chenguang Zheng, James Cheng, and George Karypis. 2023. DSP: Efficient GNN Training with Multiple GPUs. In Proceedings of the 28th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming, PPoPP 2023, Montreal, QC, Canada, 25 February 2023 - 1 March 2023. 392\u2013404."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i04.5747"},{"key":"e_1_2_1_11_1","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020","author":"Chen Ming","year":"2020","unstructured":"Ming Chen, Zhewei Wei, Bolin Ding, Yaliang Li, Ye Yuan, Xiaoyong Du, and Ji-Rong Wen. 2020. Scalable Graph Neural Networks via Bidirectional Propagation. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\u201312, 2020, virtual."},{"key":"e_1_2_1_12_1","volume-title":"Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174","author":"Chen Tianqi","year":"2016","unstructured":"Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. 2016. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174 (2016)."},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019","author":"Chiang Wei-Lin","year":"2019","unstructured":"Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, and Cho-Jui Hsieh. 2019. Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4\u20138, 2019. 257\u2013266."},{"key":"e_1_2_1_14_1","volume-title":"Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016","author":"Defferrard Micha\u00ebl","year":"2016","unstructured":"Micha\u00ebl Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5\u201310, 2016, Barcelona, Spain. 3837\u20133845."},{"key":"e_1_2_1_15_1","volume-title":"Graph Neural Networks for Social Recommendation. In The World Wide Web Conference, WWW 2019","author":"Fan Wenqi","year":"2019","unstructured":"Wenqi Fan, Yao Ma, Qing Li, Yuan He, Yihong Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph Neural Networks for Social Recommendation. In The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13\u201317, 2019. 417\u2013426."},{"key":"e_1_2_1_16_1","doi-asserted-by":"crossref","first-page":"119617","DOI":"10.1016\/j.ins.2023.119617","article-title":"Search for deep graph neural networks","volume":"649","author":"Feng Guosheng","year":"2023","unstructured":"Guosheng Feng, Hongzhi Wang, and Chunnan Wang. 2023. Search for deep graph neural networks. Inf. Sci. 649 (2023), 119617.","journal-title":"Inf. Sci."},{"key":"e_1_2_1_17_1","volume-title":"Graph Random Neural Networks for Semi-Supervised Learning on Graphs. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020","author":"Feng Wenzheng","year":"2020","unstructured":"Wenzheng Feng, Jie Zhang, Yuxiao Dong, Yu Han, Huanbo Luan, Qian Xu, Qiang Yang, Evgeny Kharlamov, and Jie Tang. 2020. Graph Random Neural Networks for Semi-Supervised Learning on Graphs. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\u201312, 2020, virtual."},{"key":"e_1_2_1_18_1","volume-title":"Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017","author":"Fout Alex","year":"2017","unstructured":"Alex Fout, Jonathon Byrd, Basir Shariat, and Asa Ben-Hur. 2017. Protein Interface Prediction using Graph Convolutional Networks. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4\u20139, 2017, Long Beach, CA, USA. 6530\u20136539."},{"key":"e_1_2_1_19_1","volume-title":"15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021","author":"Gandhi Swapnil","year":"2021","unstructured":"Swapnil Gandhi and Anand Padmanabha Iyer. 2021. P3: Distributed Deep Graph Learning at Scale. In 15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021, July 14\u201316, 2021. 551\u2013568."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.14778\/3489496.3489504"},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017","author":"Hamaguchi Takuo","year":"2017","unstructured":"Takuo Hamaguchi, Hidekazu Oiwa, Masashi Shimbo, and Yuji Matsumoto. 2017. Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19\u201325, 2017. 1802\u20131808."},{"key":"e_1_2_1_22_1","first-page":"52","article-title":"Representation Learning on Graphs","volume":"40","author":"Hamilton William L.","year":"2017","unstructured":"William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Representation Learning on Graphs: Methods and Applications. IEEE Data Eng. Bull. 40, 3 (2017), 52\u201374.","journal-title":"Methods and Applications. IEEE Data Eng. Bull."},{"key":"e_1_2_1_23_1","volume-title":"Inductive Representation Learning on Large Graphs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017","author":"Hamilton William L.","year":"2017","unstructured":"William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4\u20139, 2017, Long Beach, CA, USA. 1024\u20131034."},{"key":"e_1_2_1_24_1","volume-title":"Revisited. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022","author":"He Mingguo","year":"2022","unstructured":"Mingguo He, Zhewei Wei, and Ji-Rong Wen. 2022. Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28-December 9, 2022."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401063"},{"key":"e_1_2_1_26_1","volume-title":"Open Graph Benchmark: Datasets for Machine Learning on Graphs. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020","author":"Hu Weihua","year":"2020","unstructured":"Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open Graph Benchmark: Datasets for Machine Learning on Graphs. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\u201312, 2020, virtual."},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the Third Conference on Machine Learning and Systems, MLSys 2020","author":"Jia Zhihao","year":"2020","unstructured":"Zhihao Jia, Sina Lin, Mingyu Gao, Matei Zaharia, and Alex Aiken. 2020. Improving the Accuracy, Scalability, and Performance of Graph Neural Networks with Roc. In Proceedings of the Third Conference on Machine Learning and Systems, MLSys 2020, Austin, TX, USA, March 2\u20134, 2020."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.117921"},{"key":"e_1_2_1_29_1","volume-title":"Kipf and Max Welling","author":"Thomas","year":"2017","unstructured":"Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24\u201326, 2017, Conference Track Proceedings."},{"key":"e_1_2_1_30_1","volume-title":"7th International Conference on Learning Representations, ICLR 2019","author":"Klicpera Johannes","year":"2019","unstructured":"Johannes Klicpera, Aleksandar Bojchevski, and Stephan G\u00fcnnemann. 2019. Predict then Propagate: Graph Neural Networks meet Personalized PageRank. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6\u20139, 2019."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2898361"},{"key":"e_1_2_1_32_1","volume-title":"2019 IEEE\/CVF International Conference on Computer Vision, ICCV 2019","author":"Li Guohao","year":"2019","unstructured":"Guohao Li, Matthias M\u00fcller, Ali K. Thabet, and Bernard Ghanem. 2019. Deep-GCNs: Can GCNs Go As Deep As CNNs?. In 2019 IEEE\/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. 9266\u20139275."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.14778\/3648160.3648176"},{"key":"e_1_2_1_34_1","volume-title":"SoCC'20: ACM Symposium on Cloud Computing","author":"Lin Zhiqi","year":"2020","unstructured":"Zhiqi Lin, Cheng Li, Youshan Miao, Yunxin Liu, and Yinlong Xu. 2020. PaGraph: Scaling GNN training on large graphs via computation-aware caching. In SoCC'20: ACM Symposium on Cloud Computing, Virtual Event, USA, October 19\u201321, 2020. 401\u2013415."},{"key":"e_1_2_1_35_1","volume-title":"Towards Deeper Graph Neural Networks. In KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","author":"Liu Meng","year":"2020","unstructured":"Meng Liu, Hongyang Gao, and Shuiwang Ji. 2020. Towards Deeper Graph Neural Networks. In KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23\u201327, 2020. 338\u2013348."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3543507.3583381"},{"key":"e_1_2_1_37_1","volume-title":"Machine learning in chemoinformatics and drug discovery. Drug discovery today 23, 8","author":"Lo Yu-Chen","year":"2018","unstructured":"Yu-Chen Lo, Stefano E Rensi, Wen Torng, and Russ B Altman. 2018. Machine learning in chemoinformatics and drug discovery. Drug discovery today 23, 8 (2018), 1538\u20131546."},{"key":"e_1_2_1_38_1","volume-title":"NeuGraph: Parallel Deep Neural Network Computation on Large Graphs. In 2019 USENIX Annual Technical Conference, USENIX ATC 2019","author":"Ma Lingxiao","year":"2019","unstructured":"Lingxiao Ma, Zhi Yang, Youshan Miao, Jilong Xue, Ming Wu, Lidong Zhou, and Yafei Dai. 2019. NeuGraph: Parallel Deep Neural Network Computation on Large Graphs. In 2019 USENIX Annual Technical Conference, USENIX ATC 2019, Renton, WA, USA, July 10\u201312, 2019. 443\u2013458."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1159"},{"key":"e_1_2_1_40_1","volume-title":"Scattering GCN: Overcoming Oversmoothness in Graph Convolutional Networks. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020","author":"Min Yimeng","year":"2020","unstructured":"Yimeng Min, Frederik Wenkel, and Guy Wolf. 2020. Scattering GCN: Overcoming Oversmoothness in Graph Convolutional Networks. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\u201312, 2020, virtual."},{"key":"e_1_2_1_41_1","volume-title":"Entropy Aware Message Passing in Graph Neural Networks. arXiv preprint arXiv:2403.04636","author":"Nazari Philipp","year":"2024","unstructured":"Philipp Nazari, Oliver Lemke, Davide Guidobene, and Artiom Gesp. 2024. Entropy Aware Message Passing in Graph Neural Networks. arXiv preprint arXiv:2403.04636 (2024)."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186005"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.14778\/3538598.3538614"},{"key":"e_1_2_1_44_1","volume-title":"DropEdge: Towards Deep Graph Convolutional Networks on Node Classification. In 8th International Conference on Learning Representations, ICLR 2020","author":"Rong Yu","year":"2020","unstructured":"Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. 2020. DropEdge: Towards Deep Graph Convolutional Networks on Node Classification. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26\u201330, 2020."},{"key":"e_1_2_1_45_1","volume-title":"Collective classification in network data. AI magazine 29, 3","author":"Sen Prithviraj","year":"2008","unstructured":"Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data. AI magazine 29, 3 (2008), 93\u201393."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.14778\/3685800.3685844"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.14778\/3681954.3681976"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.3025110"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cell.2020.01.021"},{"key":"e_1_2_1_50_1","doi-asserted-by":"crossref","first-page":"803","DOI":"10.1007\/s12559-022-10098-0","article-title":"GLASS","volume":"15","author":"Sun Dengdi","year":"2023","unstructured":"Dengdi Sun, Liang Liu, Bin Luo, and Zhuanlian Ding. 2023. GLASS: A Graph Laplacian Autoencoder with Subspace Clustering Regularization for Graph Clustering. Cogn. Comput. 15, 3 (2023), 803\u2013821.","journal-title":"Cogn. Comput."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.14778\/3641204.3641219"},{"key":"e_1_2_1_52_1","volume-title":"15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021","author":"Thorpe John","year":"2021","unstructured":"John Thorpe, Yifan Qiao, Jonathan Eyolfson, Shen Teng, Guanzhou Hu, Zhihao Jia, Jinliang Wei, Keval Vora, Ravi Netravali, Miryung Kim, and Guoqing Harry Xu. 2021. Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads. In 15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021, July 14\u201316, 2021. 495\u2013514."},{"key":"e_1_2_1_53_1","volume-title":"15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021","author":"Thorpe John","year":"2021","unstructured":"John Thorpe, Yifan Qiao, Jonathan Eyolfson, Shen Teng, Guanzhou Hu, Zhihao Jia, Jinliang Wei, Keval Vora, Ravi Netravali, Miryung Kim, and Guoqing Harry Xu. 2021. Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads. In 15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021, July 14\u201316, 2021. 495\u2013514."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/SC41405.2020.00074"},{"key":"e_1_2_1_55_1","volume-title":"International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2021","author":"Vasimuddin Md.","year":"2021","unstructured":"Md. Vasimuddin, Sanchit Misra, Guixiang Ma, Ramanarayan Mohanty, Evangelos Georganas, Alexander Heinecke, Dhiraj D. Kalamkar, Nesreen K. Ahmed, and Sasikanth Avancha. 2021. DistGNN: scalable distributed training for large-scale graph neural networks. In International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2021, St. Louis, Missouri, USA, November 14\u201319, 2021. 76."},{"key":"e_1_2_1_56_1","volume-title":"Graph Attention Networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings.","author":"Velickovic Petar","year":"2018","unstructured":"Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Li\u00f2, and Yoshua Bengio. 2018. Graph Attention Networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings."},{"key":"e_1_2_1_57_1","volume-title":"Proceedings of the Twenty-Fifth International Conference (ICML 2008), Helsinki, Finland, June 5\u20139, 2008 (ACM International Conference Proceeding Series)","volume":"307","author":"Vincent Pascal","year":"2008","unstructured":"Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In Machine Learning, Proceedings of the Twenty-Fifth International Conference (ICML 2008), Helsinki, Finland, June 5\u20139, 2008 (ACM International Conference Proceeding Series), Vol. 307. 1096\u20131103."},{"key":"e_1_2_1_58_1","volume-title":"Proceedings of the Eighteenth European Conference on Computer Systems, EuroSys 2023","author":"Waleffe Roger","year":"2023","unstructured":"Roger Waleffe, Jason Mohoney, Theodoros Rekatsinas, and Shivaram Venkataraman. 2023. MariusGNN: Resource-Efficient Out-of-Core Training of Graph Neural Networks. In Proceedings of the Eighteenth European Conference on Computer Systems, EuroSys 2023, Rome, Italy, May 8\u201312, 2023. ACM, 144\u2013161."},{"key":"e_1_2_1_59_1","volume-title":"PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communication. In The Tenth International Conference on Learning Representations, ICLR 2022","author":"Wan Cheng","year":"2022","unstructured":"Cheng Wan, Youjie Li, Cameron R. Wolfe, Anastasios Kyrillidis, Nam Sung Kim, and Yingyan Lin. 2022. PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communication. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25\u201329, 2022."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589288"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2019.00070"},{"key":"e_1_2_1_62_1","volume-title":"Deep Graph Library: Towards Efficient and Scalable Deep Learning on Graphs. arXiv preprint arXiv:1909.01315","author":"Wang Minjie","year":"2019","unstructured":"Minjie Wang, Lingfan Yu, Da Zheng, Quan Gan, Yu Gai, Zihao Ye, Mufei Li, Jinjing Zhou, Qi Huang, Chao Ma, Ziyue Huang, Qipeng Guo, Hao Zhang, Haibin Lin, Junbo Zhao, Jinyang Li, Alexander J. Smola, and Zheng Zhang. 2019. Deep Graph Library: Towards Efficient and Scalable Deep Learning on Graphs. arXiv preprint arXiv:1909.01315 (2019)."},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626733"},{"key":"e_1_2_1_64_1","volume-title":"NeutronStar: Distributed GNN Training with Hybrid Dependency Management. In SIGMOD '22: International Conference on Management of Data","author":"Wang Qiange","year":"2022","unstructured":"Qiange Wang, Yanfeng Zhang, Hao Wang, Chaoyi Chen, Xiaodong Zhang, and Ge Yu. 2022. NeutronStar: Distributed GNN Training with Hybrid Dependency Management. In SIGMOD '22: International Conference on Management of Data, Philadelphia, PA, USA, June 12 - 17, 2022. 1301\u20131315."},{"key":"e_1_2_1_65_1","unstructured":"Yuke Wang Boyuan Feng Zheng Wang Tong Geng Kevin J. Barker Ang Li and Yufei Ding. 2023. MGG: Accelerating Graph Neural Networks with Fine-Grained Intra-Kernel Communication-Computation Pipelining on Multi-GPU Platforms. In 17th USENIX Symposium on Operating Systems Design and Implementation OSDI 2023 Boston MA USA July 10\u201312 2023. 779\u2013795."},{"key":"e_1_2_1_66_1","volume-title":"Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9\u201315","volume":"97","author":"Wu Felix","year":"2019","unstructured":"Felix Wu, Amauri H. Souza Jr., Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Q. Weinberger. 2019. Simplifying Graph Convolutional Networks. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9\u201315 June 2019, Long Beach, California, USA (Proceedings of Machine Learning Research), Vol. 97. 6861\u20136871."},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3535101"},{"key":"e_1_2_1_68_1","volume-title":"Hashing-Accelerated Graph Neural Networks for Link Prediction. In WWW '21: The Web Conference 2021","author":"Wu Wei","year":"2021","unstructured":"Wei Wu, Bin Li, Chuan Luo, and Wolfgang Nejdl. 2021. Hashing-Accelerated Graph Neural Networks for Link Prediction. In WWW '21: The Web Conference 2021, Virtual Event \/ Ljubljana, Slovenia, April 19\u201323, 2021. 2910\u20132920."},{"key":"e_1_2_1_69_1","volume-title":"Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks. In IEEE International Conference on Data Mining, ICDM 2022","author":"Yan Yujun","year":"2022","unstructured":"Yujun Yan, Milad Hashemi, Kevin Swersky, Yaoqing Yang, and Danai Koutra. 2022. Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks. In IEEE International Conference on Data Mining, ICDM 2022, Orlando, FL, USA, November 28-Dec. 1, 2022. 1287\u20131292."},{"key":"e_1_2_1_70_1","volume-title":"Abdelzaher","author":"Yang Chaoqi","year":"2020","unstructured":"Chaoqi Yang, Ruijie Wang, Shuochao Yao, Shengzhong Liu, and Tarek F. Abdelzaher. 2020. Revisiting \"Over-smoothing\" in Deep GCNs. CoRR abs\/2003.13663 (2020)."},{"key":"e_1_2_1_71_1","volume-title":"SC22: International Conference for High Performance Computing, Networking, Storage and Analysis","author":"Yang Dongxu","year":"2022","unstructured":"Dongxu Yang, Junhong Liu, Jiaxing Qi, and Junjie Lai. 2022. WholeGraph: A Fast Graph Neural Network Training Framework with Multi-GPU Distributed Shared Memory Architecture. In SC22: International Conference for High Performance Computing, Networking, Storage and Analysis, Dallas, TX, USA, November 13\u201318, 2022. 54:1\u201354:14."},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/3492321.3519557"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33017370"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219890"},{"key":"e_1_2_1_75_1","volume-title":"Link Prediction Based on Graph Neural Networks. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018","author":"Zhang Muhan","year":"2018","unstructured":"Muhan Zhang and Yixin Chen. 2018. Link Prediction Based on Graph Neural Networks. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3\u20138, 2018, Montr\u00e9al, Canada. 5171\u20135181."},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589311"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.14778\/3611479.3611483"},{"key":"e_1_2_1_78_1","volume-title":"PairNorm: Tackling Oversmoothing in GNNs. In 8th International Conference on Learning Representations, ICLR 2020","author":"Zhao Lingxiao","year":"2020","unstructured":"Lingxiao Zhao and Leman Akoglu. 2020. PairNorm: Tackling Oversmoothing in GNNs. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26\u201330, 2020."},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.14778\/3514061.3514069"},{"key":"e_1_2_1_80_1","volume-title":"DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs. In 10th IEEE\/ACM Workshop on Irregular Applications: Architectures and Algorithms, IA3 2020","author":"Zheng Da","year":"2020","unstructured":"Da Zheng, Chao Ma, Minjie Wang, Jinjing Zhou, Qidong Su, Xiang Song, Quan Gan, Zheng Zhang, and George Karypis. 2020. DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs. In 10th IEEE\/ACM Workshop on Irregular Applications: Architectures and Algorithms, IA3 2020, Atlanta, GA, USA, November 11, 2020. 36\u201344."},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.14778\/3598581.3598595"},{"key":"e_1_2_1_82_1","volume-title":"Understanding and Resolving Performance Degradation in Deep Graph Convolutional Networks. In CIKM '21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event","author":"Zhou Kuangqi","year":"2021","unstructured":"Kuangqi Zhou, Yanfei Dong, Kaixin Wang, Wee Sun Lee, Bryan Hooi, Huan Xu, and Jiashi Feng. 2021. Understanding and Resolving Performance Degradation in Deep Graph Convolutional Networks. In CIKM '21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 - 5, 2021. 2728\u20132737."},{"key":"e_1_2_1_83_1","volume-title":"Simple Spectral Graph Convolution. In 9th International Conference on Learning Representations, ICLR 2021","author":"Zhu Hao","year":"2021","unstructured":"Hao Zhu and Piotr Koniusz. 2021. Simple Spectral Graph Convolution. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3\u20137, 2021."},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.14778\/3352063.3352127"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3725688.3725700","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,29]],"date-time":"2025-08-29T14:21:01Z","timestamp":1756477261000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3725688.3725700"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2]]},"references-count":84,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,2]]}},"alternative-id":["10.14778\/3725688.3725700"],"URL":"https:\/\/doi.org\/10.14778\/3725688.3725700","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2025,2]]},"assertion":[{"value":"2025-08-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}