{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T16:49:23Z","timestamp":1783442963377,"version":"3.54.6"},"reference-count":77,"publisher":"Association for Computing Machinery (ACM)","issue":"4","funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2023YFB4503601"],"award-info":[{"award-number":["2023YFB4503601"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100017052","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62461146205, U2241212"],"award-info":[{"award-number":["62461146205, U2241212"]}],"id":[{"id":"10.13039\/100017052","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Distinguished Youth Foundation of Liaoning Province","award":["2024021148-JH3\/501"],"award-info":[{"award-number":["2024021148-JH3\/501"]}]},{"name":"Liaoning Province Key R&D Program Project","award":["2019JH2\/10100027"],"award-info":[{"award-number":["2019JH2\/10100027"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2025,9,22]]},"abstract":"<jats:p>Distributed training is critical for scaling graph neural networks (GNNs) to large graphs. However, existing systems struggle with efficient GNN training in heterogeneous clusters due to two main challenges. First, mapping GNN workloads to heterogeneous clusters involves matching computation and communication heterogeneity while minimizing communication overhead, resulting in a non-trivial multi-constrained multi-way workload mapping problem. Second, the positive correlation between communication and computational workloads of GNN makes workload mapping difficult, especially in heterogeneous clusters with asymmetric communication bandwidth and computational power (e.g., high-end GPUs with low-bandwidth networks).<\/jats:p>\n                  <jats:p>In this paper, we present NeutronHeter, an efficient GNN training system for heterogeneous clusters. First, we adopt a multi-level workload mapping framework. It converts the original multi-constrained multi-way workload mapping into a more efficient top-down workload mapping on a tree-like resource graph, which is constructed using hierarchical clustering based on computing power and bandwidth. By iteratively mapping the workload layer by layer, our design achieves a near-optimal solution with low overhead. Second, we adopt an adaptive communication migration strategy that reduces communication over slow connections by replicating critical remote vertices on these connections and managing their communication through faster connections. This approach significantly reduces high communication overhead in asymmetric heterogeneous clusters with low-bandwidth links and high computational power. Experimental results in heterogeneous clusters show that NeutronHeter achieves a speedup ranging from 1.06x to 33.05x compared to SOTA frameworks.<\/jats:p>","DOI":"10.1145\/3749175","type":"journal-article","created":{"date-parts":[[2025,9,23]],"date-time":"2025-09-23T17:17:03Z","timestamp":1758647823000},"page":"1-29","source":"Crossref","is-referenced-by-count":2,"title":["NeutronHeter: Optimizing Distributed Graph Neural Network Training for Heterogeneous Clusters"],"prefix":"10.1145","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-8485-1302","authenticated-orcid":false,"given":"Chunyu","family":"Cao","sequence":"first","affiliation":[{"name":"Northeastern University, Shenyang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-0746-8222","authenticated-orcid":false,"given":"Xin","family":"Ai","sequence":"additional","affiliation":[{"name":"Northeastern University, Shenyang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4847-6070","authenticated-orcid":false,"given":"Qiange","family":"Wang","sequence":"additional","affiliation":[{"name":"Northeastern University, Shenyang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9871-0304","authenticated-orcid":false,"given":"Yanfeng","family":"Zhang","sequence":"additional","affiliation":[{"name":"Northeastern University, Shenyang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-2317-9561","authenticated-orcid":false,"given":"Zhenbo","family":"Fu","sequence":"additional","affiliation":[{"name":"Northeastern University, Shenyang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-6502-7696","authenticated-orcid":false,"given":"Hao","family":"Yuan","sequence":"additional","affiliation":[{"name":"Northeastern University, Shenyang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-9646-764X","authenticated-orcid":false,"given":"Mingyi","family":"Cao","sequence":"additional","affiliation":[{"name":"Northeastern University, Shenyang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5518-9978","authenticated-orcid":false,"given":"Chaoyi","family":"Chen","sequence":"additional","affiliation":[{"name":"Northeastern University, Shenyang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6659-1785","authenticated-orcid":false,"given":"Yingyou","family":"Wen","sequence":"additional","affiliation":[{"name":"Neusoft AI Magic Technology Research, Shenyang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7422-6254","authenticated-orcid":false,"given":"Yu","family":"Gu","sequence":"additional","affiliation":[{"name":"Northeastern University, Shenyang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3171-8889","authenticated-orcid":false,"given":"Ge","family":"Yu","sequence":"additional","affiliation":[{"name":"Northeastern University, Shenyang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,9,23]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Shreyan Gupta, Hongzi Mao, and Mohammad Alizadeh.","author":"Addanki Ravichandra","year":"2019","unstructured":"Ravichandra Addanki, Shaileshh Bojja Venkatakrishnan, Shreyan Gupta, Hongzi Mao, and Mohammad Alizadeh. 2019. Placeto: Learning Generalizable Device Placement Algorithms for Distributed Machine Learning. CoRR, Vol. abs\/1906.08879 (2019). http:\/\/arxiv.org\/abs\/1906.08879"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.14778\/3659437.3659453"},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the Fifth Conference on Machine Learning and Systems, MLSys 2022","author":"Barham Paul","year":"2022","unstructured":"Paul Barham, Aakanksha Chowdhery, Jeff Dean, Sanjay Ghemawat, Steven Hand, Dan Hurt, Michael Isard, Hyeontaek Lim, Ruoming Pang, Sudip Roy, Brennan Saeta, Parker Schuh, Ryan Sepassi, Laurent El Shafey, Chandramohan A. Thekkath, and Yonghui Wu. 2022. Pathways: Asynchronous Distributed Dataflow for ML. In Proceedings of the Fifth Conference on Machine Learning and Systems, MLSys 2022, Santa Clara, CA, USA, August 29 - September 1, 2022, Diana Marculescu, Yuejie Chi, and Carole-Jean Wu (Eds.). mlsys.org. https:\/\/proceedings.mlsys.org\/paper_files\/paper\/2022\/hash\/37385144cac01dff38247ab11c119e3c-Abstract.html"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0169-7552(98)00110-X"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447786.3456233"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2391229.2391232"},{"key":"e_1_2_1_7_1","volume-title":"Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016","author":"Defferrard Micha\u00ebl","year":"2016","unstructured":"Micha\u00ebl Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (Eds.). 3837-3845."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER51413.2022.00043"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313488"},{"key":"e_1_2_1_10_1","volume-title":"Proceedings of the 19th Design Automation Conference, DAC '82","author":"Charles","year":"1982","unstructured":"Charles M. Fiduccia and Robert M. Mattheyses. 1982. A linear-time heuristic for improving network partitions. In Proceedings of the 19th Design Automation Conference, DAC '82, Las Vegas, Nevada, USA, June 14-16, 1982, James S. Crabbe, Charles E. Radke, and Hillel Ofek (Eds.). ACM\/IEEE, 175-181."},{"key":"e_1_2_1_11_1","first-page":"551","volume-title":"15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021","author":"Gandhi Swapnil","year":"2021","unstructured":"Swapnil Gandhi and Anand Padmanabha Iyer. 2021a. P3: Distributed Deep Graph Learning at Scale. In 15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021, July 14-16, 2021, Angela Demke Brown and Jay R. Lorch (Eds.). USENIX Association, 551-568. https:\/\/www.usenix.org\/conference\/osdi21\/presentation\/gandhi"},{"key":"e_1_2_1_12_1","first-page":"551","volume-title":"15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021","author":"Gandhi Swapnil","year":"2021","unstructured":"Swapnil Gandhi and Anand Padmanabha Iyer. 2021b. P3: Distributed Deep Graph Learning at Scale. In 15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021, July 14-16, 2021. 551-568."},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of the 34th International Conference on Machine Learning, ICML 2017","author":"Gilmer Justin","year":"2017","unstructured":"Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. 2017. Neural Message Passing for Quantum Chemistry. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017 (Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). PMLR, 1263-1272. http:\/\/proceedings.mlr.press\/v70\/gilmer17a.html"},{"key":"e_1_2_1_14_1","volume-title":"Inductive Representation Learning on Large Graphs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017","author":"Hamilton William L.","year":"2017","unstructured":"William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 1024-1034. https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/5dd9db5e033da9c6fb5ba83c7a7ebea9-Abstract.html"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/224170.224228"},{"key":"e_1_2_1_16_1","volume-title":"Open Graph Benchmark: Datasets for Machine Learning on Graphs. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020","author":"Hu Weihua","year":"2020","unstructured":"Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open Graph Benchmark: Datasets for Machine Learning on Graphs. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/fb60d411a5c5b72b2e7d3527cfc84fd0-Abstract.html"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3627703.3650063"},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the Third Conference on Machine Learning and Systems, MLSys 2020","author":"Jia Zhihao","year":"2020","unstructured":"Zhihao Jia, Sina Lin, Mingyu Gao, Matei Zaharia, and Alex Aiken. 2020. Improving the Accuracy, Scalability, and Performance of Graph Neural Networks with Roc. In Proceedings of the Third Conference on Machine Learning and Systems, MLSys 2020, Austin, TX, USA, March 2-4, 2020, Inderjit S. Dhillon, Dimitris S. Papailiopoulos, and Vivienne Sze (Eds.). mlsys.org. https:\/\/proceedings.mlsys.org\/paper_files\/paper\/2020\/hash\/91fc23ceccb664ebb0cf4257e1ba9c51-Abstract.html"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3035933"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1137\/S1064827595287997"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1970.tb01770.x"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/PACT.2015.15"},{"key":"e_1_2_1_23_1","volume-title":"Kipf and Max Welling","author":"Thomas","year":"2017","unstructured":"Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net. https:\/\/openreview.net\/forum?id=SJU4ayYgl"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772751"},{"key":"e_1_2_1_25_1","volume-title":"Building Meta's GenAI Infrastructure. Engineering at Meta (March","author":"Lee Kevin","year":"2024","unstructured":"Kevin Lee, Adi Gangidi, and Mathew Oldham. 2024. Building Meta's GenAI Infrastructure. Engineering at Meta (March 2024). https:\/\/engineering.fb.com\/2024\/03\/12\/data-center-engineering\/building-metas-genai-infrastructure\/"},{"key":"e_1_2_1_26_1","volume-title":"GraphTheta: A Distributed Graph Neural Network Learning System With Flexible Training Strategy. CoRR","author":"Li Houyi","year":"2021","unstructured":"Houyi Li, Yongchao Liu, Yongyong Li, Bin Huang, Peng Zhang, Guowei Zhang, Xintan Zeng, Kefeng Deng, Wenguang Chen, and Changhua He. 2021. GraphTheta: A Distributed Graph Neural Network Learning System With Flexible Training Strategy. CoRR, Vol. abs\/2104.10569 (2021). https:\/\/arxiv.org\/abs\/2104.10569"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.14778\/3648160.3648176"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3419111.3421281"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3419111.3421281"},{"key":"e_1_2_1_30_1","first-page":"443","volume-title":"Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019","author":"Ma Lingxiao","year":"2019","unstructured":"Lingxiao Ma, Zhi Yang, Youshan Miao, Jilong Xue, Ming Wu, Lidong Zhou, and Yafei Dai. 2019. NeuGraph: Parallel Deep Neural Network Computation on Large Graphs. In Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019, Renton, WA, USA, July 10-12, 2019, Dahlia Malkhi and Dan Tsafrir (Eds.). USENIX Association, 443-458. https:\/\/www.usenix.org\/conference\/atc19\/presentation\/ma"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2016.92"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783381"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2406.01566"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.48786\/edbt.2025.14"},{"key":"e_1_2_1_35_1","volume-title":"The Easiest Hard Problem: Number Partitioning","author":"Mertens Stephan","unstructured":"Stephan Mertens. 2006. The Easiest Hard Problem: Number Partitioning. In Computational Complexity and Statistical Physics, Allon G. Percus, Gabriel Istrate, and Cristopher Moore (Eds.). Oxford University Press, 125-140."},{"key":"e_1_2_1_36_1","volume-title":"Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017","author":"Monti Federico","year":"2017","unstructured":"Federico Monti, Michael M. Bronstein, and Xavier Bresson. 2017. Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 3697-3707. https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/2eace51d8f796d04991c831a07059758-Abstract.html"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCBB.2021.3060430"},{"key":"e_1_2_1_38_1","unstructured":"NVIDIA. 2024a. NVIDIA A10 Tensor Core GPU. https:\/\/www.nvidia.com\/en-us\/data-center\/products\/a10-gpu\/ Accessed: 2024-10-25."},{"key":"e_1_2_1_39_1","unstructured":"NVIDIA. 2024b. NVIDIA Tesla T4 GPU. https:\/\/www.nvidia.com\/en-us\/data-center\/tesla-t4\/ Accessed: 2024-10-25."},{"key":"e_1_2_1_40_1","unstructured":"NVIDIA. 2024c. NVIDIA Tesla T4 GPU. https:\/\/images.nvidia.com\/content\/pdf\/tesla\/184457-Tesla-P4-Datasheet-NV-Final-Letter-Web.pdf Accessed: 2024-10-25."},{"key":"e_1_2_1_41_1","volume-title":"Proceedings of the Fifth Conference on Machine Learning and Systems, MLSys 2022","author":"Or Andrew","year":"2022","unstructured":"Andrew Or, Haoyu Zhang, and Michael None Freedman. 2022. VirtualFlow: Decoupling Deep Learning Models from the Underlying Hardware. In Proceedings of the Fifth Conference on Machine Learning and Systems, MLSys 2022, Santa Clara, CA, USA, August 29 - September 1, 2022, Diana Marculescu, Yuejie Chi, and Carole-Jean Wu (Eds.). mlsys.org. https:\/\/proceedings.mlsys.org\/paper_files\/paper\/2022\/hash\/7c47b303273905755d3e513ab43ef94f-Abstract.html"},{"key":"e_1_2_1_42_1","first-page":"307","volume-title":"Proceedings of the 2020 USENIX Annual Technical Conference, USENIX ATC 2020","author":"Park Jay H.","year":"2020","unstructured":"Jay H. Park, Gyeongchan Yun, Chang M. Yi, Nguyen T. Nguyen, Seungmin Lee, Jaesik Choi, Sam H. Noh, and Young-ri Choi. 2020. HetPipe: Enabling Large DNN Training on (Whimpy) Heterogeneous GPU Clusters through Integration of Pipelined Model Parallelism and Data Parallelism. In Proceedings of the 2020 USENIX Annual Technical Conference, USENIX ATC 2020, July 15-17, 2020, Ada Gavrilovska and Erez Zadok (Eds.). USENIX Association, 307-321. https:\/\/www.usenix.org\/conference\/atc20\/presentation\/park"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2023\/724"},{"key":"e_1_2_1_44_1","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems","author":"Ren Jie","year":"2020","unstructured":"Jie Ren, Minjia Zhang, and Dong Li. 2020. HM-ANN: Efficient Billion-Point Nearest Neighbor Search on Heterogeneous Memory. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.)."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.14778\/3685800.3685844"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.14778\/3681954.3681976"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3656019.3676892"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2339530.2339722"},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of the First Workshop on Graph Neural Networks and Systems (GNNSys'21)","author":"Su Qidong","year":"2021","unstructured":"Qidong Su, Minjie Wang, Da Zheng, and Zheng Zhang. 2021. Adaptive Load Balancing for Parallel GNN Training. In Proceedings of the First Workshop on Graph Neural Networks and Systems (GNNSys'21). San Jose, CA, USA. https:\/\/gnnsys.github.io\/papers\/GNNSys21_paper_18.pdf"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.14778\/3641204.3641219"},{"key":"e_1_2_1_51_1","first-page":"495","volume-title":"15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021","author":"Thorpe John","year":"2021","unstructured":"John Thorpe, Yifan Qiao, Jonathan Eyolfson, Shen Teng, Guanzhou Hu, Zhihao Jia, Jinliang Wei, Keval Vora, Ravi Netravali, Miryung Kim, and Guoqing Harry Xu. 2021. Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads. In 15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021, July 14-16, 2021, Angela Demke Brown and Jay R. Lorch (Eds.). USENIX Association, 495-514. https:\/\/www.usenix.org\/conference\/osdi21\/presentation\/thorpe"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/SC41405.2020.00074"},{"key":"e_1_2_1_53_1","volume-title":"6th International Conference on Learning Representations, ICLR","author":"Velickovic Petar","year":"2018","unstructured":"Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Li\u00f2, and Yoshua Bengio. 2018. Graph Attention Networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. https:\/\/openreview.net\/forum?id=rJXMpikCZ"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3552326.3567501"},{"key":"e_1_2_1_55_1","volume-title":"Proceedings of the Fifth Conference on Machine Learning and Systems, MLSys 2022","author":"Wan Cheng","year":"2022","unstructured":"Cheng Wan, Youjie Li, Ang Li, Nam Sung Kim, and Yingyan Lin. 2022a. BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Partition-Parallelism and Random Boundary Node Sampling. In Proceedings of the Fifth Conference on Machine Learning and Systems, MLSys 2022, Santa Clara, CA, USA, August 29 - September 1, 2022, Diana Marculescu, Yuejie Chi, and Carole-Jean Wu (Eds.). mlsys.org. https:\/\/proceedings.mlsys.org\/paper_files\/paper\/2022\/hash\/676638b91bc90529e09b22e58abb01d6-Abstract.html"},{"key":"e_1_2_1_56_1","volume-title":"PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communication. In The Tenth International Conference on Learning Representations, ICLR 2022","author":"Wan Cheng","year":"2022","unstructured":"Cheng Wan, Youjie Li, Cameron R. Wolfe, Anastasios Kyrillidis, Nam Sung Kim, and Yingyan Lin. 2022b. PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communication. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. https:\/\/openreview.net\/forum?id=kSwqMH0zn1F"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589288"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE55515.2023.00049"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626733"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3526134"},{"key":"e_1_2_1_61_1","unstructured":"Yuke Wang Boyuan Feng Zheng Wang Tong Geng Kevin J. Barker Ang Li and Yufei Ding. 2023c. MGG: Accelerating Graph Neural Networks with Fine-Grained Intra-Kernel Communication-Computation Pipelining on Multi-GPU Platforms. In 17th USENIX Symposium on Operating Systems Design and Implementation OSDI 2023 Boston MA USA July 10-12 2023 Roxana Geambasu and Ed Nightingale (Eds.). USENIX Association 779-795. https:\/\/www.usenix.org\/conference\/osdi23\/presentation\/wang-yuke"},{"key":"e_1_2_1_62_1","volume-title":"7th International Conference on Learning Representations, ICLR 2019","author":"Xu Keyulu","year":"2019","unstructured":"Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks?. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net."},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2014.2377743"},{"key":"e_1_2_1_64_1","volume-title":"SC22: International Conference for High Performance Computing, Networking","author":"Yang Dongxu","year":"2022","unstructured":"Dongxu Yang, Junhong Liu, Jiaxing Qi, and Junjie Lai. 2022a. WholeGraph: A Fast Graph Neural Network Training Framework with Multi-GPU Distributed Shared Memory Architecture. In SC22: International Conference for High Performance Computing, Networking, Storage and Analysis, Dallas, TX, USA, November 13-18, 2022. 54:1-54:14."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3673038.3673095"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3492321.3519557"},{"key":"e_1_2_1_67_1","first-page":"264","article-title":"RAGraph","volume":"17","author":"Yao Feng","year":"2023","unstructured":"Feng Yao, Qian Tao, Wenyuan Yu, Yanfeng Zhang, Shufeng Gong, Qiange Wang, Ge Yu, and Jingren Zhou. 2023. RAGraph: A Region-Aware Framework for Geo-Distributed Graph Processing. Proc. VLDB Endow., Vol. 17, 3 (2023), 264-277. https:\/\/www.vldb.org\/pvldb\/vol17\/p264-yao.pdf","journal-title":"A Region-Aware Framework for Geo-Distributed Graph Processing. Proc. VLDB Endow."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219890"},{"key":"e_1_2_1_69_1","first-page":"11960","volume-title":"Graph Transformer Networks. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019","author":"Yun Seongjun","year":"2019","unstructured":"Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J. Kim. 2019. Graph Transformer Networks. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d'Alch\u00e9-Buc, Emily B. Fox, and Roman Garnett (Eds.). 11960-11970."},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3627703.3629580"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589311"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.14778\/3611479.3611483"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.14778\/3514061.3514069"},{"key":"e_1_2_1_74_1","first-page":"36","volume-title":"DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs. In 10th IEEE\/ACM Workshop on Irregular Applications: Architectures and Algorithms, IA3 2020","author":"Zheng Da","year":"2020","unstructured":"Da Zheng, Chao Ma, Minjie Wang, Jinjing Zhou, Qidong Su, Xiang Song, Quan Gan, Zheng Zhang, and George Karypis. 2020. DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs. In 10th IEEE\/ACM Workshop on Irregular Applications: Architectures and Algorithms, IA3 2020, Atlanta, GA, USA, November 11, 2020. 36-44."},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539177"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.14778\/3352063.3352127"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.14778\/3352063.3352127"}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3749175","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T04:34:21Z","timestamp":1781325261000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3749175"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,22]]},"references-count":77,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,9,22]]}},"alternative-id":["10.1145\/3749175"],"URL":"https:\/\/doi.org\/10.1145\/3749175","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,22]]}}}