{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T16:46:37Z","timestamp":1782405997201,"version":"3.54.5"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T00:00:00Z","timestamp":1782345600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/legalcode"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62272190 and 62302190"],"award-info":[{"award-number":["62272190 and 62302190"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Key Research and Development Program of China","award":["2023YFB4502304"],"award-info":[{"award-number":["2023YFB4502304"]}]},{"DOI":"10.13039\/501100013061","name":"Jilin Scientific and Technological Development Program","doi-asserted-by":"crossref","award":["20250102205JC"],"award-info":[{"award-number":["20250102205JC"]}],"id":[{"id":"10.13039\/501100013061","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100007847","name":"Natural Science Foundation of Jilin Province","doi-asserted-by":"crossref","award":["20230101062JC"],"award-info":[{"award-number":["20230101062JC"]}],"id":[{"id":"10.13039\/100007847","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Liaoning Natural Science Funds","award":["2025-BS-0212"],"award-info":[{"award-number":["2025-BS-0212"]}]},{"DOI":"10.13039\/501100017683","name":"Dalian Science and Technology Innovation Fund","doi-asserted-by":"crossref","award":["2024JB11GX001"],"award-info":[{"award-number":["2024JB11GX001"]}],"id":[{"id":"10.13039\/501100017683","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>Distributed Graph Neural Network (DGNN) is a powerful tool in large-scale graph representation learning. However, high data-transfer overhead among workers in a DGNN training job confines its scalability and thus the overall performance. Vertex-wise historical embedding methods have demonstrated high potential to alleviate the problems, but still suffer from severe accuracy loss and limited performance scalability, which has been attributed to the information loss of critical channels in historical vertices and redundant information in local channels. This article explores the optimization of channel level and construct a quantitative accuracy model for channel-wise historical embedding. We propose Sift, a novel DGNN training framework, supporting channel-wise partial historical embedding with accuracy guarantee. Sift has three components: a historical embedding evaluator with channel-wise quantitative accuracy model, a sawtooth-like matrix rearrangement for accelerating message passing, and a hybrid parallel framework for overlapping communication overhead. Comprehensive experimental results show that Sift achieves near-linear parallel convergence speedup, outperforming the state-of-the-art baselines by up to 72% in total training performance and up to 21% in convergence speed.<\/jats:p>","DOI":"10.1145\/3809161","type":"journal-article","created":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T11:30:25Z","timestamp":1778499025000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Sift: Channel-Wise Historical Embedding for High Efficiency Distributed Graph Neural Network Training with Accuracy Guarantee"],"prefix":"10.1145","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8309-6084","authenticated-orcid":false,"given":"Zhewen","family":"Xu","sequence":"first","affiliation":[{"name":"College of Computer Science and Technology, Jilin University","place":["Changchun, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4377-0550","authenticated-orcid":false,"given":"Hongliang","family":"Li","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Jilin University","place":["Changchun, China"]},{"name":"Jilin University Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education","place":["Changchun, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-1007-3938","authenticated-orcid":false,"given":"Junze","family":"Han","sequence":"additional","affiliation":[{"name":"First Automobile Works Group Corp","place":["Changchun, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2189-8385","authenticated-orcid":false,"given":"Hengshan","family":"Yue","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Jilin University","place":["Changchun, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-7081-5172","authenticated-orcid":false,"given":"Hairui","family":"Zhao","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Jilin University","place":["Changchun, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-3031-389X","authenticated-orcid":false,"given":"Dongyuan","family":"Tian","sequence":"additional","affiliation":[{"name":"Jilin University","place":["Changchun, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3901-6790","authenticated-orcid":false,"given":"Zijian","family":"Li","sequence":"additional","affiliation":[{"name":"Dalian Maritime University","place":["Dalian, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5597-3625","authenticated-orcid":false,"given":"Xiaohui","family":"Wei","sequence":"additional","affiliation":[{"name":"Jilin University","place":["Changchun, China"]},{"name":"Jilin University Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education","place":["Changchun, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,25]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3532611"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-024-00795-w"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.14778\/3538598.3538614"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/IA351965.2020.00011"},{"key":"e_1_3_1_6_2","first-page":"3294","volume-title":"ICML","author":"Fey Matthias","year":"2021","unstructured":"Matthias Fey, Jan E. Lenssen, et\u00a0al. 2021. Gnnautoscale: Scalable and expressive graph neural networks via historical embeddings. In ICML. PMLR, 3294\u20133304."},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458817.3480856"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","unstructured":"Haiyang Lin Mingyu Yan Xiaochun Ye Dongrui Fan Shirui Pan Wenguang Chen Yuan Xie. 2023. A comprehensive survey on distributed training of graph neural networks. In Proceedings of the IEEE 111 12 (2023) 1572\u20131606. DOI:10.1109\/jproc.2023.3337442","DOI":"10.1109\/jproc.2023.3337442"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3526134"},{"key":"e_1_3_1_10_2","first-page":"551","volume-title":"OSDI 21","author":"Gandhi Swapnil","year":"2021","unstructured":"Swapnil Gandhi and Anand Padmanabha Iyer. 2021. P3: Distributed deep graph learning at scale. In OSDI 21. 551\u2013568."},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM48880.2022.9796910"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3648358"},{"key":"e_1_3_1_13_2","article-title":"What graph neural networks cannot learn: Depth vs width","author":"Loukas Andreas","year":"2019","unstructured":"Andreas Loukas. 2019. What graph neural networks cannot learn: Depth vs width. arXiv preprint arXiv:1907.03199 (2019).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_14_2","doi-asserted-by":"crossref","unstructured":"Ming Zhou Jie Lu et\u00a0al. 2023. Multi-stream concept drift self-adaptation using graph neural network. IEEE Transactions on Knowledge and Data Engineering 35 12 (2023) 12828\u201312841.","DOI":"10.1109\/TKDE.2023.3272911"},{"key":"e_1_3_1_15_2","article-title":"Learn locally, correct globally: A distributed algorithm for training graph neural networks","author":"Ramezani Morteza","year":"2021","unstructured":"Morteza Ramezani, Weilin Cong, Mehrdad Mahdavi, Mahmut T. Kandemir, and Anand Sivasubramaniam. 2021. Learn locally, correct globally: A distributed algorithm for training graph neural networks. arXiv preprint arXiv:2111.08202 (2021).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.3007399"},{"key":"e_1_3_1_17_2","article-title":"On the convergence of adam and beyond","author":"Reddi Sashank J.","year":"2019","unstructured":"Sashank J. Reddi, Satyen Kale, and Sanjiv Kumar. 2019. On the convergence of adam and beyond. arXiv preprint arXiv:1904.09237 (2019).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_18_2","article-title":"Stochastic training of graph convolutional networks with variance reduction","author":"Chen Jianfei","year":"2017","unstructured":"Jianfei Chen, Jun Zhu, and Le Song. 2017. Stochastic training of graph convolutional networks with variance reduction. arXiv preprint arXiv:1710.10568 (2017).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_19_2","first-page":"3837","volume-title":"Advances in Neural Information Processing Systems (NIPS\u201916)","author":"Defferrard Micha\u00ebl","year":"2016","unstructured":"Micha\u00ebl Defferrard, Xavier Bresson, et\u00a0al. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in Neural Information Processing Systems (NIPS\u201916) 29 (2016), 3837\u20133845."},{"issue":"20","key":"e_1_3_1_20_2","first-page":"10","article-title":"Graph attention networks","volume":"1050","author":"Velickovic Petar","year":"2017","unstructured":"Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio, et\u00a0al. 2017. Graph attention networks. Stat 1050, 20 (2017), 10\u201348550.","journal-title":"Stat"},{"key":"e_1_3_1_21_2","article-title":"Semi-supervised classification with graph convolutional networks","author":"Kipf Thomas N.","year":"2016","unstructured":"Thomas N. Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.adi2336"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1038\/35019019"},{"key":"e_1_3_1_24_2","first-page":"17","volume-title":"OSDI 12","author":"Gonzalez Joseph E.","year":"2012","unstructured":"Joseph E. Gonzalez et\u00a0al. 2012. PowerGraph: Distributed Graph-Parallel computation on natural graphs. In OSDI 12. 17\u201330."},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3572848.3577528"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330925"},{"key":"e_1_3_1_27_2","article-title":"MG-GCN: Fast and effective learning with mix-grained aggregators for training large graph convolutional networks","author":"Huang Tao","year":"2020","unstructured":"Tao Huang, Yihan Zhang, et\u00a0al. 2020. MG-GCN: Fast and effective learning with mix-grained aggregators for training large graph convolutional networks. arXiv preprint arXiv:2011.09900 (2020).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_28_2","first-page":"1","volume-title":"SC20","author":"Tripathy Alok","year":"2020","unstructured":"Alok Tripathy, Katherine Yelick, et\u00a0al. 2020. Reducing communication in graph neural network training. In SC20. IEEE, 1\u201314."},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2023.3305077"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3419111.3421281"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3572848.3577487"},{"key":"e_1_3_1_32_2","unstructured":"Zheng Chai Guangji Bai Liang Zhao and Yue Cheng. 2022. Distributed Graph Neural Network Training with Periodic Stale Representation Synchronization. (2022). arxiv:cs.LG\/2206.00057. Retrieved from https:\/\/arxiv.org\/abs\/2206.00057"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.155"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i3.16350"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/72.265964"},{"key":"e_1_3_1_36_2","first-page":"4839","volume-title":"International Conference on Machine Learning","author":"N\u00f8kland Arild","year":"2019","unstructured":"Arild N\u00f8kland and Lars Hiller Eidnes. 2019. Training neural networks with local error signals. In International Conference on Machine Learning. PMLR, 4839\u20134850."},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP48485.2024.10447377"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1002\/advs.202203485"},{"key":"e_1_3_1_39_2","unstructured":"Chenzhong Yin Mingxi Cheng Xiongye Xiao Xinghe Chen Shahin Nazarian Andrei Irimia and Paul Bogdan. 2023. Leader-Follower Neural Networks with Local Error Signals Inspired by Complex Collectives. (2023). arxiv:cs.LG\/2310.07885. Retrieved from https:\/\/arxiv.org\/abs\/2310.07885"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.matcom.2023.11.002"},{"key":"e_1_3_1_41_2","first-page":"13","volume-title":"Learning on Graphs Conference","author":"Yang Ruochen","year":"2022","unstructured":"Ruochen Yang, Frederic Sala, and Paul Bogdan. 2022. Efficient representation learning for higher-order data with simplicial complexes. In Learning on Graphs Conference. PMLR, 13\u20131."},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-020-05929-w"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.aiopen.2021.01.001"},{"key":"e_1_3_1_44_2","article-title":"Some fundamental aspects about lipschitz continuity of neural networks","author":"Khromov Grigory","year":"2023","unstructured":"Grigory Khromov and Sidak Pal Singh. 2023. Some fundamental aspects about lipschitz continuity of neural networks. arXiv preprint arXiv:2302.10886 (2023).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.physrep.2009.05.001"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403192"},{"key":"e_1_3_1_47_2","first-page":"5453","volume-title":"ICML","author":"Xu Keyulu","year":"2018","unstructured":"Keyulu Xu, Chengtao Li, et\u00a0al. 2018. Representation learning on graphs with jumping knowledge networks. In ICML. PMLR, 5453\u20135462."},{"key":"e_1_3_1_48_2","volume-title":"The Eleventh International Conference on Learning Representations (ICLR 2023)","author":"Shi Zhihao","year":"2023","unstructured":"Zhihao Shi, Xize Liang, et\u00a0al. 2023. LMC: Fast training of GNNs via subgraph sampling with provable convergence. The Eleventh International Conference on Learning Representations (ICLR 2023)."},{"issue":"1","key":"e_1_3_1_49_2","first-page":"140","article-title":"The pareto principle","volume":"7","author":"Dunford Rosie","year":"2014","unstructured":"Rosie Dunford, Quanrong Su, et\u00a0al. 2014. The pareto principle. The Plymouth Student Scientist 7, 1 (2014), 140\u2013148.","journal-title":"The Plymouth Student Scientist"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2022.06.031"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.22436\/jnsa.008.03.08"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.csda.2007.01.003"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2013.224"},{"key":"e_1_3_1_54_2","first-page":"803","volume-title":"International Conference on Artificial Intelligence and Statistics","author":"Dutta Sanghamitra","year":"2018","unstructured":"Sanghamitra Dutta et\u00a0al. 2018. Slow and stale gradients can win the race: Error-runtime trade-offs in distributed SGD. In International Conference on Artificial Intelligence and Statistics. PMLR, 803\u2013812."},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.2978386"},{"issue":"1","key":"e_1_3_1_56_2","first-page":"515","article-title":"Semi-dynamic load balancing: Efficient distributed learning in non-dedicated environments","volume":"11","author":"Chen Chen","year":"2020","unstructured":"Chen Chen, Qizhen Weng, Baochun Li, and Bo Li. 2020. Semi-dynamic load balancing: Efficient distributed learning in non-dedicated environments. In Proceedings of the 11th ACM Symposium on Cloud Computing (SoCC\u201920) 11, 1 (2020), 515\u2013531.","journal-title":"Proceedings of the 11th ACM Symposium on Cloud Computing (SoCC\u201920)"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-57077-4_10"},{"key":"e_1_3_1_58_2","volume-title":"ICLR Workshop on Representation Learning on Graphs and Manifolds","author":"Fey Matthias","year":"2019","unstructured":"Matthias Fey et\u00a0al. 2019. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds."},{"key":"e_1_3_1_59_2","first-page":"22118","article-title":"Open graph benchmark: Datasets for machine learning on graphs","volume":"33","author":"Hu Weihua","year":"2020","unstructured":"Weihua Hu, Matthias Fey, et\u00a0al. 2020. Open graph benchmark: Datasets for machine learning on graphs. Advances in Neural Information Processing Systems 33 (2020), 22118\u201322133.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2023-078432"},{"key":"e_1_3_1_61_2","first-page":"1024","volume-title":"Advances in Neural Information Processing Systems (NIPS 2017)","author":"Hamilton William L.","year":"2017","unstructured":"William L. Hamilton, Rex Ying, et\u00a0al. 2017. Inductive representation learning on large graphs. Advances in Neural Information Processing Systems (NIPS 2017) 30 (2017), 1024\u20131034."},{"issue":"1","key":"e_1_3_1_62_2","first-page":"i47\u2013i56","article-title":"Protein function prediction via graph kernels","volume":"21","author":"Borgwardt Karsten M.","year":"2005","unstructured":"Karsten M. Borgwardt, Cheng Soon Ong, Stefan Sch\u00f6nauer, S. V. N. Vishwanathan, Alex J. Smola, and Hans-Peter Kriegel. 2005. Protein function prediction via graph kernels. Bioinformatics 21, suppl_1 (2005), i47\u2013i56.","journal-title":"Bioinformatics"},{"key":"e_1_3_1_63_2","volume-title":"ICLR Workshop on Representation Learning on Graphs and Manifolds","author":"Wang Minjie Yu","year":"2019","unstructured":"Minjie Yu Wang. 2019. Deep graph library: Towards efficient and scalable deep learning on graphs. In ICLR Workshop on Representation Learning on Graphs and Manifolds."},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkr1179"},{"key":"e_1_3_1_65_2","article-title":"Weatherbench 2: A benchmark for the next generation of data-driven global weather models","author":"Rasp Stephan","year":"2023","unstructured":"Stephan Rasp et\u00a0al. 2023. Weatherbench 2: A benchmark for the next generation of data-driven global weather models. arXiv preprint arXiv:2308.15560 (2023).","journal-title":"arXiv preprint"}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3809161","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T15:56:16Z","timestamp":1782402976000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3809161"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,25]]},"references-count":64,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3809161"],"URL":"https:\/\/doi.org\/10.1145\/3809161","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"value":"1544-3566","type":"print"},{"value":"1544-3973","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,25]]},"assertion":[{"value":"2024-07-02","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-04","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-25","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}