{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T22:42:03Z","timestamp":1779230523656,"version":"3.51.4"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2025,2,15]],"date-time":"2025-02-15T00:00:00Z","timestamp":1739577600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["92270114"],"award-info":[{"award-number":["92270114"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"University Synergy Innovation Program of Anhui Province","award":["GXXT-2022-040"],"award-info":[{"award-number":["GXXT-2022-040"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2025,2,28]]},"abstract":"<jats:p>\n            Graph augmentation is the key component to reveal instance-discriminative features of a graph as its rationale\u2014an interpretation for it\u2014in graph contrastive learning (GCL). Existing rationale-aware augmentation mechanisms in GCL frameworks roughly fall into two categories and suffer from inherent limitations: (1) non-heuristic methods with the guidance of domain knowledge to preserve salient features, which require expensive expertise and lack generality, or (2) heuristic augmentations with a co-trained auxiliary model to identify crucial substructures, which face not only the dilemma between system complexity and transformation diversitybut also the instability stemming from the co-training of two separated sub-models. Inspired by recent studies on transformers, we propose self-attentive rationale-guided GCL (SR-GCL), which integrates rationale generator and encoder together, leverages the self-attention values in transformer module as a natural guidance to delineate semantically informative substructures from both node- and edge-wise perspectives, and contrasts on rationale-aware augmented pairs. On real-world biochemistry datasets, visualization results verify the effectiveness and interpretability of self-attentive rationalization, and the performance on downstream tasks demonstrates the state-of-the-art performance of SR-GCL for graph model pre-training. Codes are available at\n            <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/lsh0520\/SR-GCL\">https:\/\/github.com\/lsh0520\/SR-GCL<\/jats:ext-link>\n            .\n          <\/jats:p>","DOI":"10.1145\/3665894","type":"journal-article","created":{"date-parts":[[2024,5,23]],"date-time":"2024-05-23T15:51:51Z","timestamp":1716479511000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Self-attentive Rationalization for Interpretable Graph Contrastive Learning"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-8986-7965","authenticated-orcid":false,"given":"Sihang","family":"Li","sequence":"first","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-2637-176X","authenticated-orcid":false,"given":"Yanchen","family":"Luo","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1367-711X","authenticated-orcid":false,"given":"An","family":"Zhang","sequence":"additional","affiliation":[{"name":"National University of Singapore, Singapore, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6148-6329","authenticated-orcid":false,"given":"Xiang","family":"Wang","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9263-7011","authenticated-orcid":false,"given":"Longfei","family":"Li","sequence":"additional","affiliation":[{"name":"Ant Group, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6033-6102","authenticated-orcid":false,"given":"Jun","family":"Zhou","sequence":"additional","affiliation":[{"name":"Ant Group, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6097-7807","authenticated-orcid":false,"given":"Tat-Seng","family":"Chua","sequence":"additional","affiliation":[{"name":"National University of Singapore, Singapore, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,2,15]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"1448","volume-title":"Proceedings of International Conference on Machine Learning Research (PMLR)","volume":"119","author":"Chang Shiyu","year":"2020","unstructured":"Shiyu Chang, Yang Zhang, Mo Yu, and Tommi S. Jaakkola. 2020. Invariant Rationalization. In Proceedings of International Conference on Machine Learning Research (PMLR), Vol. 119. 1448\u20131458."},{"key":"e_1_3_2_3_2","first-page":"3469","volume-title":"Proceedings of International Conference on Machine Learning Research (PMLR)","volume":"162","author":"Chen Dexiong","year":"2022","unstructured":"Dexiong Chen, Leslie O\u2019Bray, and Karsten M. Borgwardt. 2022. Structure-Aware Transformer for Graph Representation Learning. In Proceedings of International Conference on Machine Learning Research (PMLR), Vol. 162. 3469\u20133489."},{"key":"e_1_3_2_4_2","first-page":"1597","volume-title":"Proceedings of International Conference on Machine Learning Research (PMLR)","volume":"119","author":"Chen Ting","year":"2020","unstructured":"Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. 2020. A Simple Framework for Contrastive Learning of Visual Representations. In Proceedings of International Conference on Machine Learning Research (PMLR), Vol. 119. 1597\u20131607."},{"key":"e_1_3_2_5_2","first-page":"4171","volume-title":"NAACL-HLT (1). Association for Computational Linguistics","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT (1). Association for Computational Linguistics, 4171\u20134186."},{"key":"e_1_3_2_6_2","volume-title":"International Conference on Learning Representations (ICLR)","author":"Dosovitskiy Alexey","year":"2021","unstructured":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Conference on Learning Representations (ICLR). Retrieved from https:\/\/openreview.net\/"},{"key":"e_1_3_2_7_2","unstructured":"Vijay Prakash Dwivedi and Xavier Bresson. 2020. A Generalization of Transformer Networks to Graphs. arXiv:2012.09699v2. Retrieved from https:\/\/arxiv.org\/abs\/2012.09699"},{"key":"e_1_3_2_8_2","first-page":"1024","volume-title":"Advances in Neural Information Processing Systems (NIPS)","author":"Hamilton William L.","year":"2017","unstructured":"William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In Advances in Neural Information Processing Systems (NIPS). 1024\u20131034."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01553"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539321"},{"key":"e_1_3_2_11_2","volume-title":"International Conference on Learning Representations","author":"Hu Weihua","year":"2020","unstructured":"Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay S. Pande, and Jure Leskovec. 2020. Strategies for Pre-training Graph Neural Networks. In International Conference on Learning Representations."},{"key":"e_1_3_2_12_2","volume-title":"International Conference on Learning Representations (ICLR \u201917)","author":"Jang Eric","year":"2017","unstructured":"Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical Reparametrization with Gumble-Softmax. In International Conference on Learning Representations (ICLR \u201917). Retrieved from https:\/\/openreview.net\/"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.5555\/3540261.3541915"},{"key":"e_1_3_2_14_2","unstructured":"Greg Landrum. 2010. RDkit Open-Source Cheminformatics Software. Retrieved from https:\/\/www.rdkit.org\/"},{"key":"e_1_3_2_15_2","unstructured":"Sihang Li Zhiyuan Liu Yanchen Luo Xiang Wang Xiangnan He Kenji Kawaguchi Tat-Seng Chua and Qi Tian. 2024. Towards 3D Molecule-Text Interpretation in Language Models. arXiv:2401.13923v2. Retrieved from https:\/\/arxiv.org\/abs\/2401.13923"},{"key":"e_1_3_2_16_2","first-page":"13052","volume-title":"Proceedings of International Conference on Machine Learning Research (PMLR)","volume":"162","author":"Li Sihang","year":"2022","unstructured":"Sihang Li, Xiang Wang, An Zhang, Yingxin Wu, Xiangnan He, and Tat-Seng Chua. 2022. Let Invariant Rationale Discovery Inspire Graph Contrastive Learning. In Proceedings of International Conference on Machine Learning Research (PMLR), Vol. 162. 13052\u201313065."},{"key":"e_1_3_2_17_2","volume-title":"International Conference on Learning Representations (ICLR)","author":"Liu Shengchao","year":"2022","unstructured":"Shengchao Liu, Hanchen Wang, Weiyang Liu, Joan Lasenby, Hongyu Guo, and Jian Tang. 2022. Pre-Training Molecular Graph Representation with 3D Geometry. In International Conference on Learning Representations (ICLR). Retrieved from https:\/\/openreview.net\/"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.966"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE60146.2024.00203"},{"key":"e_1_3_2_20_2","unstructured":"Gr\u00e9goire Mialon Dexiong Chen Margot Selosse and Julien Mairal. 2021. GraphiT: Encoding Graph Structure in Transformers. arXiv:2106.05667v1. Retrieved from https:\/\/arxiv.org\/abs\/2106.05667"},{"key":"e_1_3_2_21_2","unstructured":"Christopher Morris Nils M. Kriege Franka Bause Kristian Kersting Petra Mutzel and Marion Neumann. 2020. TUDataset: A Collection of Benchmark Datasets for Learning with Graphs. arXiv:2007.08663v1. Retrieved from https:\/\/arxiv.org\/abs\/2007.08663"},{"key":"e_1_3_2_22_2","unstructured":"Annamalai Narayanan Mahinthan Chandramohan Rajasekar Venkatesan Lihui Chen Yang Liu and Shantanu Jaiswal. 2017. graph2vec: Learning Distributed Representations of Graphs. arXiv:1707.05005. Retrieved from https:\/\/arxiv.org\/abs\/1707.05005 (2017)."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403168"},{"key":"e_1_3_2_24_2","unstructured":"Ladislav Ramp\u00e1sek Mikhail Galkin Vijay Prakash Dwivedi Anh Tuan Luu Guy Wolf and Dominique Beaini. 2022. Recipe for a general powerful scalable graph transformer. arXiv:2205.12454. Retrieved from https:\/\/arxiv.org\/abs\/2205.12454"},{"key":"e_1_3_2_25_2","volume-title":"Advances in Neural Information Processing Systems (NeurIPS)","author":"Rong Yu","year":"2020","unstructured":"Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. 2020. Self-Supervised Graph Transformer on Large-Scale Molecular Data. In Advances in Neural Information Processing Systems (NeurIPS)."},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.366"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1021\/acs.jcim.5b00559"},{"key":"e_1_3_2_28_2","unstructured":"Fan-Yun Sun Jordan Hoffmann Vikas Verma and Jian Tang. 2020. InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization. In International Conference on Learning Representations (ICLR)."},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.5555\/3540261.3541479"},{"key":"e_1_3_2_30_2","first-page":"37","article-title":"Feature Selection for Classification: A Review","author":"Tang Jiliang","year":"2014","unstructured":"Jiliang Tang, Salem Alelyani, and Huan Liu. 2014. Feature Selection for Classification: A Review. In Data Classification: Algorithms and Applications. CRC Press. 37\u201364.","journal-title":"Data Classification: Algorithms and Applications"},{"key":"e_1_3_2_31_2","unstructured":"Josephine M. Thomas Alice Moallemy-Oureh Silvia Beddar-Wiesing and Clara Holzh\u00fcter. 2022. Graph Neural Networks Designed for Different Graph Types: A Survey. arXiv:2204.03080. Retrieved from https:\/\/arxiv.org\/abs\/2204.03080"},{"key":"e_1_3_2_32_2","unstructured":"A\u00e4ron van den Oord Yazhe Li and Oriol Vinyals. 2018. Representation Learning with Contrastive Predictive Coding. arXiv:1807.03748. Retrieved from https:\/\/arxiv.org\/abs\/1807.03748"},{"key":"e_1_3_2_33_2","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems (NIPS)","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All you Need. In Advances in Neural Information Processing Systems (NIPS). 5998\u20136008."},{"key":"e_1_3_2_34_2","unstructured":"Petar Velickovic William Fedus William L. Hamilton Pietro Lio Yoshua Bengio and R. Devon Hjelm. 2019. Deep Graph Infomax. In International Conference on Learning Representations (ICLR) (Poster). Retrieved from https:\/\/openreview.net\/"},{"key":"e_1_3_2_35_2","doi-asserted-by":"crossref","unstructured":"Feng Wang and Huaping Liu. 2021. Understanding the Behaviour of Contrastive Loss. In 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Computer Vision Foundation\/IEEE 2495\u20132504.","DOI":"10.1109\/CVPR46437.2021.00252"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2022.3178128"},{"key":"e_1_3_2_37_2","volume-title":"International Conference on Learning Representations","author":"Wang Yifei","year":"2021","unstructured":"Yifei Wang, Qi Zhang, Yisen Wang, Jiansheng Yang, and Zhouchen Lin. 2021. Chaos Is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap. In International Conference on Learning Representations."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.5555\/3540261.3541277"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1039\/C7SC02664A"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00393"},{"key":"e_1_3_2_41_2","volume-title":"International Conference on Learning Representations (ICLR)","author":"Xu Keyulu","year":"2019","unstructured":"Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks? In International Conference on Learning Representations (ICLR)."},{"key":"e_1_3_2_42_2","first-page":"11548","volume-title":"Proceedings of International Conference on Machine Learning Research (PMLR)","volume":"139","author":"Xu Minghao","year":"2021","unstructured":"Minghao Xu, Hang Wang, Bingbing Ni, Hongyu Guo, and Jian Tang. 2021. Self-Supervised Graph-Level Representation Learning with Local and Global Structure. In Proceedings of International Conference on Machine Learning Research (PMLR), Vol. 139. 11548\u201311558."},{"issue":"3","key":"e_1_3_2_43_2","first-page":"3169","article-title":"Group Contrastive Self-Supervised Learning on Graphs","volume":"45","author":"Xu Xinyi","year":"2022","unstructured":"Xinyi Xu, Cheng Deng, Yaochen Xie, and Shuiwang Ji. 2022. Group Contrastive Self-Supervised Learning on Graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 3 (2022), 3169\u20133180.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.5555\/3540261.3542473"},{"key":"e_1_3_2_45_2","first-page":"12121","volume-title":"Proceedings of International Conference on Machine Learning Research (PMLR)","volume":"139","author":"You Yuning","year":"2021","unstructured":"Yuning You, Tianlong Chen, Yang Shen, and Zhangyang Wang. 2021. Graph Contrastive Learning Automated. In Proceedings of International Conference on Machine Learning Research (PMLR), Vol. 139. 12121\u201312132."},{"key":"e_1_3_2_46_2","first-page":"5812","volume-title":"Proceedings of the 34th International Conference on Neural Information Processing Systems","volume":"33","author":"You Yuning","year":"2020","unstructured":"Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph Contrastive Learning with Agmentations. In Proceedings of the 34th International Conference on Neural Information Processing Systems, Vol. 33. 5812\u20135823."},{"key":"e_1_3_2_47_2","volume-title":"34th Conference on Neural Information Processing Systems (NeurIPS)","author":"Zaheer Manzil","year":"2020","unstructured":"Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Onta\u00f1\u00f3n, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. 2020. Big Bird: Transformers for Longer Sequences. In 34th Conference on Neural Information Processing Systems (NeurIPS)."},{"key":"e_1_3_2_48_2","unstructured":"Yanqiao Zhu Yichen Xu Feng Yu Qiang Liu Shu Wu and Liang Wang. 2020. Deep Graph Contrastive Representation Learning. arXiv:2006.04131. Retrieved from https:\/\/arxiv.org\/abs\/2006.04131"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3442381.3449802"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3665894","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3665894","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:44:27Z","timestamp":1750290267000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3665894"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,15]]},"references-count":48,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,2,28]]}},"alternative-id":["10.1145\/3665894"],"URL":"https:\/\/doi.org\/10.1145\/3665894","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,15]]},"assertion":[{"value":"2023-09-08","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-15","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}