{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,11]],"date-time":"2025-09-11T17:56:02Z","timestamp":1757613362088,"version":"3.44.0"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"9","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2025,5]]},"abstract":"<jats:p>Memory-based temporal graph neural network (MTGNN) models are effective for predicting temporal graphs by using node memory and message-passing modules to capture temporal and structural information, respectively. However, distributed training for large graphs presents challenges such as accuracy loss and decreased efficiency due to remote features and memory transmission. Despite improvements in MTGNN system optimizations, issues like dynamic load imbalances, communication overhead, and memory staleness persist. To tackle these challenges, we introduce MemShare, a distributed MTGNN system. MemShare introduces a novel shared node memory paradigm that utilizes a small subset of shared nodes across machines and GPUs to reduce distributed communication for memory management. It incorporates techniques like shared nodes-centric graph partitioning, shared nodes-aware boundary decay sampling, and shared nodes-targeted synchronous smoothing aggregation. Experiments show that MemShare outperforms existing distributed MTGNN systems in accuracy and training efficiency.<\/jats:p>","DOI":"10.14778\/3746405.3746430","type":"journal-article","created":{"date-parts":[[2025,9,3]],"date-time":"2025-09-03T17:06:20Z","timestamp":1756919180000},"page":"3093-3105","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Effective and Efficient Distributed Temporal Graph Learning through Hotspot Memory Sharing"],"prefix":"10.14778","volume":"18","author":[{"given":"Longjiao","family":"Zhang","sequence":"first","affiliation":[{"name":"Zhejiang University"}]},{"given":"Rui","family":"Wang","sequence":"additional","affiliation":[{"name":"Zhejiang University, High-Tech Zone (Binjiang), Institute of Blockchain and Data Security"}]},{"given":"Tongya","family":"Zheng","sequence":"additional","affiliation":[{"name":"Zhejiang Key Laboratory of Big Data Intelligent Computing, Hangzhou City University"}]},{"given":"Ziqi","family":"Huang","sequence":"additional","affiliation":[{"name":"Zhejiang University"}]},{"given":"Wenjie","family":"Huang","sequence":"additional","affiliation":[{"name":"Zhejiang University"}]},{"given":"Xinyu","family":"Wang","sequence":"additional","affiliation":[{"name":"Zhejiang University"}]},{"given":"Can","family":"Wang","sequence":"additional","affiliation":[{"name":"Zhejiang University"}]},{"given":"Mingli","family":"Song","sequence":"additional","affiliation":[{"name":"Zhejiang University"}]},{"given":"Sai","family":"Wu","sequence":"additional","affiliation":[{"name":"Zhejiang University"}]},{"given":"Shuibing","family":"He","sequence":"additional","affiliation":[{"name":"Zhejiang University"}]}],"member":"320","published-online":{"date-parts":[[2025,9,3]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313488"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2021.106746"},{"key":"e_1_2_1_3_1","volume-title":"Graph neural network for traffic forecasting: A survey. Expert systems with applications, 207:117921","author":"Jiang Weiwei","year":"2022","unstructured":"Weiwei Jiang and Jiayun Luo. Graph neural network for traffic forecasting: A survey. Expert systems with applications, 207:117921, 2022."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2023.3257759"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2023.122156"},{"key":"e_1_2_1_6_1","volume-title":"A review on graph neural network methods in financial applications. arXiv preprint arXiv:2111.15367","author":"Wang Jianian","year":"2021","unstructured":"Jianian Wang, Sheng Zhang, Yanghua Xiao, and Rui Song. A review on graph neural network methods in financial applications. arXiv preprint arXiv:2111.15367 2021."},{"key":"e_1_2_1_7_1","volume-title":"Temporal graph networks for deep learning on dynamic graphs. arxiv","author":"Rossi Emanuele","year":"2020","unstructured":"Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning on dynamic graphs. arxiv 2020. arXiv preprint arXiv:2006.10637."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330895"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457564"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380076"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.14778\/3529337.3529342"},{"key":"e_1_2_1_12_1","volume-title":"Speed: Streaming partition and parallel acceleration for temporal interaction graph embedding. arXiv preprint arXiv:2308.14129","author":"Chen Xi","year":"2023","unstructured":"Xi Chen, Yongxiang Liao, Yun Xiong, Yao Zhang, Siwei Zhang, Jiawei Zhang, and Yiheng Sun. Speed: Streaming partition and parallel acceleration for temporal interaction graph embedding. arXiv preprint arXiv:2308.14129, 2023."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2024.3432855"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588195.3592990"},{"key":"e_1_2_1_15_1","volume-title":"Gnnflow: A distributed framework for continuous temporal gnn learning on dynamic graphs. arXiv preprint arXiv:2311.17410","author":"Zhong Yuchen","year":"2023","unstructured":"Yuchen Zhong, Guangming Sheng, Tianzuo Qin, Minjie Wang, Quan Gan, and Chuan Wu. Gnnflow: A distributed framework for continuous temporal gnn learning on dynamic graphs. arXiv preprint arXiv:2311.17410, 2023."},{"key":"e_1_2_1_16_1","volume-title":"Proceedings of the International Conference on Database Systems for Advanced Applications","author":"Fang Ziquan","year":"2024","unstructured":"Ziquan Fang, Qichen Sun, Qilong Wang, Lu Chen, and Yunjun Gao. Distributed temporal graph neural network learning over large-scale dynamic graphs. In Proceedings of the International Conference on Database Systems for Advanced Applications, 2024."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3581784.3607056"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3637528.3671844"},{"key":"e_1_2_1_19_1","unstructured":"Wiki-talk. http:\/\/snap.stanford.edu\/data\/wiki-talk-temporal.html."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.14778\/3641204.3641215"},{"volume-title":"The Twelfth International Conference on Learning Representations.","author":"Su Junwei","key":"e_1_2_1_21_1","unstructured":"Junwei Su, Difan Zou, and Chuan Wu. Pres: Toward scalable memory-based dynamic graph neural networks. In The Twelfth International Conference on Learning Representations."},{"key":"e_1_2_1_22_1","volume-title":"Foundations and modeling of dynamic networks using dynamic graph neural networks: A survey. iEEE Access, 9:79143\u201379168","author":"Skarding Joakim","year":"2021","unstructured":"Joakim Skarding, Bogdan Gabrys, and Katarzyna Musial. Foundations and modeling of dynamic networks using dynamic graph neural networks: A survey. iEEE Access, 9:79143\u201379168, 2021."},{"key":"e_1_2_1_23_1","volume-title":"Inductive representation learning on temporal graphs. arXiv preprint arXiv:2002.07962","author":"Xu Da","year":"2020","unstructured":"Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. Inductive representation learning on temporal graphs. arXiv preprint arXiv:2002.07962, 2020."},{"key":"e_1_2_1_24_1","volume-title":"International Conference on Learning Representations (ICLR)","author":"Wang Yanbang","year":"2021","unstructured":"Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. Inductive representation learning in temporal networks via causal anonymous walks. In International Conference on Learning Representations (ICLR), 2021."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.14778\/3583140.3583150"},{"key":"e_1_2_1_26_1","first-page":"937","volume-title":"2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS)","author":"Deng Gangda","unstructured":"Gangda Deng, Hongkuan Zhou, Hanqing Zeng, Yinglong Xia, Christopher Leung, Jianbo Li, Rajgopal Kannan, and Viktor Prasanna. Taser: Temporal adaptive sampling for fast and accurate dynamic graph representation learning. In 2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 926\u2013937. IEEE, 2024."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/305219.305248"},{"key":"e_1_2_1_28_1","unstructured":"Stack-overflow. https:\/\/snap.stanford.edu\/data\/sx-stackoverflow.html."},{"key":"e_1_2_1_29_1","first-page":"49","volume-title":"Gdelt: Global data on events, location, and tone","author":"Leetaru Kalev","year":"1979","unstructured":"Kalev Leetaru and Philip A Schrodt. Gdelt: Global data on events, location, and tone, 1979\u20132012. In ISA annual convention, volume 2, pages 1\u201349. Citeseer, 2013."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/SC41406.2024.00056"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556195.2556213"},{"key":"e_1_2_1_32_1","first-page":"30","volume-title":"USENIX OSDI","author":"Gonzalez Joseph E","year":"2012","unstructured":"Joseph E Gonzalez, Yucheng Low, Haijie Gu, Danny Bickson, and Carlos Guestrin. PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs. In USENIX OSDI, pages 17\u201330, 2012."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO56248.2022.00091"},{"key":"e_1_2_1_34_1","volume-title":"Provably expressive temporal graph networks. Advances in neural information processing systems, 35:32257\u201332269","author":"Souza Amauri","year":"2022","unstructured":"Amauri Souza, Diego Mesquita, Samuel Kaski, and Vikas Garg. Provably expressive temporal graph networks. Advances in neural information processing systems, 35:32257\u201332269, 2022."},{"key":"e_1_2_1_35_1","volume-title":"Pytorch distributed: Experiences on accelerating data parallel training. arXiv preprint arXiv:2006.15704","author":"Li Shen","year":"2020","unstructured":"Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, et al. Pytorch distributed: Experiences on accelerating data parallel training. arXiv preprint arXiv:2006.15704, 2020."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3620665.3640414"},{"key":"e_1_2_1_37_1","unstructured":"NetworkX Developers. networkx-metis: Metis integration for networkx. https:\/\/github.com\/networkx\/networkx-metis."},{"key":"e_1_2_1_38_1","first-page":"14","volume-title":"SC","author":"Md Vasimuddin","year":"2021","unstructured":"Vasimuddin Md, Sanchit Misra, Guixiang Ma, Ramanarayan Mohanty, Evangelos Georganas, Alexander Heinecke, Dhiraj Kalamkar, Nesreen K Ahmed, and Sasikanth Avancha. Distgnn: Scalable distributed training for large-scale graph neural networks. In SC, pages 1\u201314, 2021."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.14778\/3514061.3514069"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3492321.3519557"},{"key":"e_1_2_1_41_1","first-page":"118","volume-title":"USENIX NSDI","author":"Liu Tianfeng","year":"2023","unstructured":"Tianfeng Liu, Yangrui Chen, Dan Li, Chuan Wu, Yibo Zhu, Jun He, and et.al. Peng. Bgl:gpu-efficient gnn training by optimizing graph data i\/o and preprocessing. In USENIX NSDI, pages 103\u2013118, 2023."},{"key":"e_1_2_1_42_1","first-page":"477","article-title":"Communication-efficient graph neural networks with probabilistic neighborhood expansion analysis and caching","volume":"5","author":"Kaler Tim","year":"2023","unstructured":"Tim Kaler, Alexandros Iliopoulos, Philip Murzynowski, Tao Schardl, Charles E Leiserson, and Jie Chen. Communication-efficient graph neural networks with probabilistic neighborhood expansion analysis and caching. Proceedings of Machine Learning and Systems, 5:477\u2013494, 2023.","journal-title":"Proceedings of Machine Learning and Systems"},{"key":"e_1_2_1_43_1","first-page":"82","volume-title":"Proceedings of the European Conference on Computer Systems","author":"Wang Lei","year":"2021","unstructured":"Lei Wang, Qiang Yin, Chao Tian, Jianbang Yang, Rong Chen, Wenyuan Yu, Zihang Yao, and Jingren Zhou. Flexgraph: a flexible and efficient distributed framework for gnn training. In Proceedings of the European Conference on Computer Systems, pages 67\u201382, 2021."},{"volume-title":"International Conference on Learning Representations.","author":"Wan Cheng","key":"e_1_2_1_44_1","unstructured":"Cheng Wan, Youjie Li, Cameron R Wolfe, Anastasios Kyrillidis, Nam Sung Kim, and Yingyan Lin. Pipegcn: Efficient full-graph training of graph convolutional networks with pipelined feature communication. In International Conference on Learning Representations."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.14778\/3632093.3632108"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3654977"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588737"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3746405.3746430","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,4]],"date-time":"2025-09-04T19:51:58Z","timestamp":1757015518000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3746405.3746430"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5]]},"references-count":47,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2025,5]]}},"alternative-id":["10.14778\/3746405.3746430"],"URL":"https:\/\/doi.org\/10.14778\/3746405.3746430","relation":{},"ISSN":["2150-8097"],"issn-type":[{"type":"print","value":"2150-8097"}],"subject":[],"published":{"date-parts":[[2025,5]]},"assertion":[{"value":"2025-09-03","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}