{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T01:49:39Z","timestamp":1775526579269,"version":"3.50.1"},"reference-count":79,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2025,1,2]],"date-time":"2025-01-02T00:00:00Z","timestamp":1735776000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Institute of Information & Communications Technology Planning & Evaluation","award":["2022-0-00157\/RS-2022-II220157"],"award-info":[{"award-number":["2022-0-00157\/RS-2022-II220157"]}]},{"name":"Robust, Fair, Extensible Data-Centric Continual Learning","award":["RS-2024-00438638"],"award-info":[{"award-number":["RS-2024-00438638"]}]},{"name":"EntireDB2AI: Foundations and Software for Comprehensive Deep Representation Learning and Prediction on Entire Relational Databases","award":["2019-0-00075 \/ RS-2019-II190075"],"award-info":[{"award-number":["2019-0-00075 \/ RS-2019-II190075"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2025,2,28]]},"abstract":"<jats:p>\n            Continual Learning (CL) is the process of learning ceaselessly a sequence of tasks. Most existing CL methods deal with independent data (e.g., images and text) for which many benchmark frameworks and results under standard experimental settings are available. Compared to them, however, CL methods for graph data (graph CL) are relatively underexplored because of (a) the lack of standard experimental settings, especially regarding how to deal with the dependency between instances, (b) the lack of benchmark datasets and scenarios, and (c) high complexity in implementation and evaluation due to the dependency. In this paper, regarding (a) we define four standard incremental settings (task-, class-, domain-, and time-incremental) for node-, link-, and graph-level problems, extending the previously explored scope. Regarding (b), we provide 35 benchmark scenarios based on 24 real-world graphs. Regarding (c), we develop\n            <jats:sc>BeGin<\/jats:sc>\n            , an easy and fool-proof framework for graph CL.\n            <jats:sc>BeGin<\/jats:sc>\n            is easily extended since it is modularized with reusable modules for data processing, algorithm design, and evaluation. Especially, the evaluation module is completely separated from user code to eliminate potential mistakes. Regarding benchmark results, we cover\n            <jats:inline-formula content-type=\"math\/tex\">\n              <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(3\\times\\)<\/jats:tex-math>\n            <\/jats:inline-formula>\n            more combinations of incremental settings and levels of problems than the latest benchmark. All assets for the benchmark framework are publicly available at\n            <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"url\" xlink:href=\"https:\/\/github.com\/ShinhwanKang\/BeGin\">https:\/\/github.com\/ShinhwanKang\/BeGin<\/jats:ext-link>\n            .\n          <\/jats:p>","DOI":"10.1145\/3702648","type":"journal-article","created":{"date-parts":[[2024,11,1]],"date-time":"2024-11-01T05:27:37Z","timestamp":1730438857000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["BeGin: Extensive Benchmark Scenarios and an Easy-to-use Framework for Graph Continual Learning"],"prefix":"10.1145","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3223-8318","authenticated-orcid":false,"given":"Jihoon","family":"Ko","sequence":"first","affiliation":[{"name":"Kim Jaechul Graduate School of AI, KAIST, Seoul, South Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6434-1347","authenticated-orcid":false,"given":"Shinhwan","family":"Kang","sequence":"additional","affiliation":[{"name":"Kim Jaechul Graduate School of AI, KAIST, Seoul, South Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6177-7329","authenticated-orcid":false,"given":"Taehyung","family":"Kwon","sequence":"additional","affiliation":[{"name":"Kim Jaechul Graduate School of AI, KAIST, Seoul, South Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7086-8911","authenticated-orcid":false,"given":"Heechan","family":"Moon","sequence":"additional","affiliation":[{"name":"Kim Jaechul Graduate School of AI, KAIST, Seoul, South Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2872-1526","authenticated-orcid":false,"given":"Kijung","family":"Shin","sequence":"additional","affiliation":[{"name":"Kim Jaechul Graduate School of AI, KAIST, Seoul, South Korea"}]}],"member":"320","published-online":{"date-parts":[[2025,1,2]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2012.120"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01219-9_9"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3455345"},{"key":"e_1_3_2_5_2","first-page":"546","volume-title":"Proceedings of the 34th International Conference on Neural Information Processing Systems","author":"Baek Jinheon","year":"2020","unstructured":"Jinheon Baek, Dong Bok Lee, and Sung Ju Hwang. 2020. Learning to extrapolate knowledge: Transductive few-shot out-of-graph link prediction. In Proceedings of the 34th International Conference on Neural Information Processing Systems, 546\u2013560."},{"key":"e_1_3_2_6_2","unstructured":"Aleksandar Bojchevski and Stephan G\u00fcnnemann. 2017. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. arXiv:1707.03815. Retrieved from https:\/\/arxiv.org\/abs\/1707.03815"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485447.3512176"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2302.08191"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","unstructured":"Antonio Carta Andrea Cossu Federico Errica and Davide Bacciu. 2021. Catastrophic forgetting in deep graph networks: an introductory benchmark for graph classification. arXiv:2103.11750. Retrieved from 10.48550\/arXiv.2103.11750","DOI":"10.48550\/arXiv.2103.11750"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01252-6_33"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","unstructured":"Arslan Chaudhry Marcus Rohrbach Mohamed Elhoseiny Thalaiyasingam Ajanthan Puneet K Dokania Philip H. S. Torr and Marc\u2019Aurelio Ranzato. 2019. On tiny episodic memories in continual learning. arXiv:1902.10486. Retrieved from 10.48550\/arXiv.1902.10486","DOI":"10.48550\/arXiv.1902.10486"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330925"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3056071"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","unstructured":"Vijay Prakash Dwivedi Chaitanya K. Joshi Anh Tuan Luu Thomas Laurent Yoshua Bengio and Xavier Bresson. 2020. Benchmarking graph neural networks. arXiv:2003.00982. Retrieved from 10.48550\/arXiv.2003.00982","DOI":"10.48550\/arXiv.2003.00982"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","unstructured":"Beyza Ermis Giovanni Zappella Martin Wistuba and Cedric Archambeau. 2022. Memory efficient continual learning for neural text classification. arXiv:2203.04640. Retrieved from 10.48550\/arXiv.2203.04640","DOI":"10.48550\/arXiv.2203.04640"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2022.3222049"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN52387.2021.9533412"},{"issue":"12","key":"e_1_3_2_18_2","first-page":"2009","article-title":"Twitter sentiment classification using distant supervision","volume":"1","author":"Go Alec","year":"2009","unstructured":"Alec Go, Richa Bhayani, and Lei Huang. 2009. Twitter sentiment classification using distant supervision. CS224N Project Report, Stanford 1, 12 (2009), 2009.","journal-title":"CS224N Project Report, Stanford"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1021\/acscentsci.7b00572"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","unstructured":"Ian J. Goodfellow Mehdi Mirza Da Xiao Aaron Courville and Yoshua Bengio. 2013. An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv:1312.6211. Retrieved from 10.48550\/arXiv.1312.6211","DOI":"10.48550\/arXiv.1312.6211"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2827872"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1186\/s42467-021-00009-8"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","unstructured":"Matthew Honnibal Ines Montani Sofie Van Landeghem and Adriane Boyd. 2020. spaCy: Industrial-Strength Natural Language Processing in Python. DOI: 10.5281\/zenodo.1212303","DOI":"10.5281\/zenodo.1212303"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","unstructured":"Yen-Chang Hsu Yen-Cheng Liu Anita Ramasamy and Zsolt Kira. 2018. Re-evaluating continual learning scenarios: A categorization and case for strong baselines. arXiv:1810.12488. Retrieved from 10.48550\/arXiv.1810.12488","DOI":"10.48550\/arXiv.1810.12488"},{"key":"e_1_3_2_25_2","first-page":"22118","volume-title":"Proceedings of the 34th International Conference on Neural Information Processing Systems","author":"Hu Weihua","year":"2020","unstructured":"Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open graph benchmark: Datasets for machine learning on graphs. In Proceedings of the 34th International Conference on Neural Information Processing Systems, 22118\u201322133."},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1038\/nmicrobiol.2016.48"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","unstructured":"Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980. Retrieved from 10.48550\/arXiv.1412.6980","DOI":"10.48550\/arXiv.1412.6980"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","unstructured":"Thomas N. Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv:1609.02907. Retrieved from 10.48550\/arXiv.1609.02907","DOI":"10.48550\/arXiv.1609.02907"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1611835114"},{"key":"e_1_3_2_30_2","unstructured":"Jihoon Ko Shinhwan Kang Taehyung Kwon Heechan Moon and Kijung Shin. 2022. Begin: Extensive benchmark scenarios and an easy-to-use framework for graph continual learning. arXiv:2211.14568. Retrieved from https:\/\/arxiv.org\/abs\/2211.14568"},{"key":"e_1_3_2_31_2","unstructured":"Jihoon Ko Shinhwan Kang Taehyung Kwon Heechan Moon and Kijung Shin. 2024. The Official Website of the BeGin Framework. Retrieved from https:\/\/github.com\/ShinhwanKang\/BeGin"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","unstructured":"Jihoon Ko Taehyung Kwon Kijung Shin and Juho Lee. 2021. Learning to pool in graph neural networks for extrapolation. arXiv:2106.06210. Retrieved from 10.48550\/arXiv.2106.06210","DOI":"10.48550\/arXiv.2106.06210"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159729"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2016.0033"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","unstructured":"Greg Landrum. 2006. RDKit: Open-Source Cheminformatics. DOI: 10.5281\/zenodo.591637","DOI":"10.5281\/zenodo.591637"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2022.3193725"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1606.09282"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883090"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2201.06289"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i10.17049"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM58522.2023.00141"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1705.03550"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW53098.2021.00399"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1706.08840"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01225-0_5"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00810"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","unstructured":"P\u00e9ter Mernyei and C\u0103t\u0103lina Cangea. 2020. Wiki-cs: A wikipedia-based benchmark for graph neural networks. arXiv:2007.02901. DOI: 10.48550\/arXiv.2007.02901","DOI":"10.48550\/arXiv.2007.02901"},{"key":"e_1_3_2_48_2","unstructured":"NYC Taxi & Limousine Commission. 2023. TLC Trip Record Data. Retrieved from https:\/\/www.nyc.gov\/site\/tlc\/about\/tlc-trip-record-data.page."},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3018661.3018731"},{"key":"e_1_3_2_50_2","first-page":"16131","volume-title":"Proceedings of the 35th International Conference on Neural Information Processing Systems","author":"Pham Quang","year":"2021","unstructured":"Quang Pham, Chenghao Liu, and Steven Hoi. 2021. Dualnet: Continual learning, fast and slow. In Proceedings of the 35th International Conference on Neural Information Processing Systems, 16131\u201316144."},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.587"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","unstructured":"Emanuele Rossi Ben Chamberlain Fabrizio Frasca Davide Eynard Federico Monti and Michael Bronstein. 2020. Temporal graph networks for deep learning on dynamic graphs. arXiv:2006.10637. Retrieved from 10.48550\/arXiv.2006.10637","DOI":"10.48550\/arXiv.2006.10637"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341161.3342890"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","unstructured":"Benedek Rozemberczki and Rik Sarkar. 2021. Twitch gamers: A dataset for evaluating proximity preserving and structural role-based node embeddings. arXiv:2101.03091. Retrieved from 10.48550\/arXiv.2101.03091","DOI":"10.48550\/arXiv.2101.03091"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v29i3.2157"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1801.01423"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1705.08690"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-019-0151-1"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","unstructured":"Fan-Yun Sun Jordan Hoffmann Vikas Verma and Jian Tang. 2019. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. arXiv:1908.01000. DOI: 10.48550\/arXiv.1908.01000","DOI":"10.48550\/arXiv.1908.01000"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i4.25586"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gky1131"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/2746230"},{"key":"e_1_3_2_63_2","first-page":"11","article-title":"Visualizing data using t-SNE","volume":"9","author":"Maaten Laurens Van der","year":"2008","unstructured":"Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 11 (2008), 2579\u20132605.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_64_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Veli\u010dkovi\u0107 Petar","year":"2018","unstructured":"Petar Veli\u010dkovi\u0107, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Li\u00f2, and Yoshua Bengio. 2018. Graph attention networks. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_2_65_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Veli\u010dkovi\u0107 Petar","year":"2019","unstructured":"Petar Veli\u010dkovi\u0107, William Fedus, William L. Hamilton, Pietro Li\u00f2, Yoshua Bengio, and R. Devon Hjelm. 2019. Deep graph infomax. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01335"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3411963"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1162\/qss_a_00021"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331267"},{"key":"e_1_3_2_70_2","first-page":"19435","volume-title":"Proceedings of the 35th International Conference on Neural Information Processing Systems","author":"Wu Qitian","year":"2021","unstructured":"Qitian Wu, Chenxiao Yang, and Junchi Yan. 2021. Towards open-world feature extrapolation: An inductive graph learning approach. In Proceedings of the 35th International Conference on Neural Information Processing Systems, 19435\u201319447."},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1039\/C7SC02664A"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1021\/acs.jmedchem.9b00959"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","unstructured":"Da Xu Chuanwei Ruan Evren Korpeoglu Sushant Kumar and Kannan Achan. 2020. Inductive representation learning on temporal graphs. arXiv:2002.07962. Retrieved from 10.48550\/arXiv.2002.07962","DOI":"10.48550\/arXiv.2002.07962"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539300"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2010.13902"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3591652"},{"key":"e_1_3_2_77_2","first-page":"13006","volume-title":"Proceedings of the 36th International Conference on Neural Information Processing Systems","author":"Zhang Xikun","year":"2022","unstructured":"Xikun Zhang, Dongjin Song, and Dacheng Tao. 2022. CGLB: Benchmark tasks for continual graph learning. In Proceedings of the 36th International Conference on Neural Information Processing Systems, 13006\u201313021."},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2022.3186909"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i5.16602"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1818013116"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3702648","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3702648","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:03Z","timestamp":1750295883000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3702648"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,2]]},"references-count":79,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,2,28]]}},"alternative-id":["10.1145\/3702648"],"URL":"https:\/\/doi.org\/10.1145\/3702648","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,2]]},"assertion":[{"value":"2024-02-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}