{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,26]],"date-time":"2025-03-26T00:26:17Z","timestamp":1742948777997,"version":"3.40.3"},"publisher-location":"Cham","reference-count":27,"publisher":"Springer Nature Switzerland","isbn-type":[{"type":"print","value":"9783031333767"},{"type":"electronic","value":"9783031333774"}],"license":[{"start":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T00:00:00Z","timestamp":1672531200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,5,28]],"date-time":"2023-05-28T00:00:00Z","timestamp":1685232000000},"content-version":"vor","delay-in-days":147,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Recently self-supervised learning is gaining popularity for Graph Neural Networks\u00a0(GNN) by leveraging unlabeled data. Augmentation plays a key role in self-supervision. While there is a common set of image augmentation methods that preserve image labels in general, graph augmentation methods do not guarantee consistent graph semantics and are usually domain dependent. Existing self-supervised GNN models often handpick a small set of augmentation techniques that limit the performance of the model.<\/jats:p><jats:p>In this paper, we propose a common set of graph augmentation methods to a wide range of GNN tasks, and rely on the Pareto optimality to select and balance among these possibly conflicting augmented versions, called <jats:bold>P<\/jats:bold>areto <jats:bold>G<\/jats:bold>raph <jats:bold>C<\/jats:bold>ontrastive <jats:bold>L<\/jats:bold>earning\u00a0(<jats:bold>PGCL<\/jats:bold>) framework. We show that while random selection of the same set of augmentation leads to slow convergence or even divergence, PGCL converges much faster with lower error rate. Extensive experiments on multiple datasets of different domains and scales demonstrate superior or comparable performance of PGCL.\n<\/jats:p>","DOI":"10.1007\/978-3-031-33377-4_38","type":"book-chapter","created":{"date-parts":[[2023,5,27]],"date-time":"2023-05-27T09:02:36Z","timestamp":1685178156000},"page":"495-507","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Multi-Augmentation Contrastive Learning as\u00a0Multi-Objective Optimization for\u00a0Graph Neural Networks"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-2237-0682","authenticated-orcid":false,"given":"Xu","family":"Li","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yongsheng","family":"Chen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,5,28]]},"reference":[{"key":"38_CR1","unstructured":"Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597\u20131607. PMLR (2020)"},{"key":"38_CR2","doi-asserted-by":"crossref","unstructured":"Chu, G., Wang, X., Shi, C., Jiang, X.: CUCO: graph representation with curriculum contrastive learning. In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pp. 2300\u20132306. International Joint Conferences on Artificial Intelligence Organization (8 2021)","DOI":"10.24963\/ijcai.2021\/317"},{"issue":"5\u20136","key":"38_CR3","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1016\/j.crma.2012.03.014","volume":"350","author":"JA D\u00e9sid\u00e9ri","year":"2012","unstructured":"D\u00e9sid\u00e9ri, J.A.: Multiple-gradient descent algorithm (mgda) for multiobjective optimization. C.R. Math. 350(5\u20136), 313\u2013318 (2012)","journal-title":"C.R. Math."},{"key":"38_CR4","unstructured":"Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)"},{"key":"38_CR5","doi-asserted-by":"publisher","DOI":"10.1016\/j.sigpro.2021.108310","volume":"190","author":"H Hafidi","year":"2022","unstructured":"Hafidi, H., Ghogho, M., Ciblat, P., Swami, A.: Negative sampling strategies for contrastive self-supervised learning of graph representations. Sig. Process. 190, 108310 (2022)","journal-title":"Sig. Process."},{"key":"38_CR6","unstructured":"Hassani, K., Khasahmadi, A.H.: Contrastive multi-view representation learning on graphs. In: International Conference on Machine Learning, pp. 4116\u20134126. PMLR (2020)"},{"key":"38_CR7","doi-asserted-by":"crossref","unstructured":"He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729\u20139738 (2020)","DOI":"10.1109\/CVPR42600.2020.00975"},{"key":"38_CR8","unstructured":"Hern\u00e1ndez-Lobato, D., Hernandez-Lobato, J., Shah, A., Adams, R.: Predictive entropy search for multi-objective Bayesian optimization. In: International Conference on Machine Learning, pp. 1492\u20131501. PMLR (2016)"},{"key":"38_CR9","doi-asserted-by":"crossref","unstructured":"Hou, Z., Liu, X., Dong, Y., Wang, C., Tang, J., et al.: GraphMAE: self-supervised masked graph autoencoders. arXiv preprint arXiv:2205.10803 (2022)","DOI":"10.1145\/3534678.3539321"},{"key":"38_CR10","unstructured":"Jun, H., et al.: Distribution augmentation for generative modeling. In: International Conference on Machine Learning, pp. 5006\u20135019. PMLR (2020)"},{"issue":"1","key":"38_CR11","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1109\/TNNLS.2014.2309939","volume":"26","author":"C Li","year":"2014","unstructured":"Li, C., Georgiopoulos, M., Anagnostopoulos, G.C.: Pareto-path multitask multiple kernel learning. IEEE Trans. Neural Netw. Learn. Syst. 26(1), 51\u201361 (2014)","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"38_CR12","unstructured":"Morris, C., Kriege, N.M., Bause, F., Kersting, K., Mutzel, P., Neumann, M.: Tudataset: a collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663 (2020)"},{"key":"38_CR13","unstructured":"Narayanan, A., Chandramohan, M., Venkatesan, R., Chen, L., Liu, Y., Jaiswal, S.: graph2vec: learning distributed representations of graphs. arXiv preprint arXiv:1707.05005 (2017)"},{"key":"38_CR14","unstructured":"Sener, O., Koltun, V.: Multi-task learning as multi-objective optimization. arXiv preprint arXiv:1810.04650 (2018)"},{"key":"38_CR15","unstructured":"Shervashidze, N., Schweitzer, P., Van Leeuwen, E.J., Mehlhorn, K., Borgwardt, K.M.: Weisfeiler-lehman graph kernels. Journal of Machine Learning Research 12(9) (2011)"},{"key":"38_CR16","unstructured":"Suresh, S., Li, P., Hao, C., Neville, J.: Adversarial graph augmentation to improve graph contrastive learning. In: NeurIPS (2021)"},{"key":"38_CR17","unstructured":"Tong, Z., Liang, Y., Ding, H., Dai, Y., Li, X., Wang, C.: Directed graph contrastive learning. In: Advances in Neural Information Processing Systems, vol. 34 (2021)"},{"issue":"1","key":"38_CR18","first-page":"3483","volume":"15","author":"K Van Moffaert","year":"2014","unstructured":"Van Moffaert, K., Now\u00e9, A.: Multi-objective reinforcement learning using sets of pareto dominating policies. J. Mach. Learn. Res. 15(1), 3483\u20133512 (2014)","journal-title":"J. Mach. Learn. Res."},{"issue":"3","key":"38_CR19","first-page":"4","volume":"2","author":"P Velickovic","year":"2019","unstructured":"Velickovic, P., Fedus, W., Hamilton, W.L., Li\u00f2, P., Bengio, Y., Hjelm, R.D.: Deep graph infomax. ICLR (Poster) 2(3), 4 (2019)","journal-title":"ICLR (Poster)"},{"key":"38_CR20","doi-asserted-by":"crossref","unstructured":"Wang, Y., Wang, W., Liang, Y., Cai, Y., Liu, J., Hooi, B.: Nodeaug: semi-supervised node classification with data augmentation. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 207\u2013217 (2020)","DOI":"10.1145\/3394486.3403063"},{"key":"38_CR21","unstructured":"Hu, W., et al.: Strategies for pre-training graph neural networks. In: International Conference on Learning Representations (2020). https:\/\/openreview.net\/forum?id=HJlWWJSFDH"},{"key":"38_CR22","unstructured":"Xu, D., Cheng, W., Luo, D., Chen, H., Zhang, X.: InfoGCL: information-aware graph contrastive learning. In: Advances in Neural Information Processing Systems, vol. 34 (2021)"},{"key":"38_CR23","unstructured":"Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018)"},{"key":"38_CR24","doi-asserted-by":"crossref","unstructured":"Yanardag, P., Vishwanathan, S.: Deep graph kernels. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1365\u20131374 (2015)","DOI":"10.1145\/2783258.2783417"},{"key":"38_CR25","unstructured":"You, Y., Chen, T., Shen, Y., Wang, Z.: Graph contrastive learning automated. In: Proceedings of the 38th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 139, pp. 12121\u201312132. PMLR (18\u201324 Jul 2021). https:\/\/proceedings.mlr.press\/v139\/you21a.html"},{"key":"38_CR26","first-page":"5812","volume":"33","author":"Y You","year":"2020","unstructured":"You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., Shen, Y.: Graph contrastive learning with augmentations. Adv. Neural. Inf. Process. Syst. 33, 5812\u20135823 (2020)","journal-title":"Adv. Neural. Inf. Process. Syst."},{"key":"38_CR27","doi-asserted-by":"crossref","unstructured":"You, Y., Chen, T., Wang, Z., Shen, Y.: Bringing your own view: graph contrastive learning without prefabricated data augmentations. In: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp. 1300\u20131309 (2022)","DOI":"10.1145\/3488560.3498416"}],"container-title":["Lecture Notes in Computer Science","Advances in Knowledge Discovery and Data Mining"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-031-33377-4_38","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,16]],"date-time":"2023-06-16T12:06:47Z","timestamp":1686917207000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-031-33377-4_38"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"ISBN":["9783031333767","9783031333774"],"references-count":27,"URL":"https:\/\/doi.org\/10.1007\/978-3-031-33377-4_38","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"type":"print","value":"0302-9743"},{"type":"electronic","value":"1611-3349"}],"subject":[],"published":{"date-parts":[[2023]]},"assertion":[{"value":"28 May 2023","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"PAKDD","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Pacific-Asia Conference on Knowledge Discovery and Data Mining","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Osaka","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Japan","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2023","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"25 May 2023","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"28 May 2023","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"27","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"pakdd2023","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/pakdd2023.org\/","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Double-blind","order":1,"name":"type","label":"Type","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"Microsoft CMT","order":2,"name":"conference_management_system","label":"Conference Management System","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"813","order":3,"name":"number_of_submissions_sent_for_review","label":"Number of Submissions Sent for Review","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"143","order":4,"name":"number_of_full_papers_accepted","label":"Number of Full Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"0","order":5,"name":"number_of_short_papers_accepted","label":"Number of Short Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"18% - The value is computed by the equation \"Number of Full Papers Accepted \/ Number of Submissions Sent for Review * 100\" and then rounded to a whole number.","order":6,"name":"acceptance_rate_of_full_papers","label":"Acceptance Rate of Full Papers","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3.5","order":7,"name":"average_number_of_reviews_per_paper","label":"Average Number of Reviews per Paper","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"10","order":8,"name":"average_number_of_papers_per_reviewer","label":"Average Number of Papers per Reviewer","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"Yes","order":9,"name":"external_reviewers_involved","label":"External Reviewers Involved","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}}]}}