{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T16:02:49Z","timestamp":1774540969368,"version":"3.50.1"},"reference-count":53,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T00:00:00Z","timestamp":1720742400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T00:00:00Z","timestamp":1720742400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Johann Wolfgang Goethe-Universit\u00e4t, Frankfurt am Main"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2024,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Graph Neural Networks (GNNs) are a popular class of machine learning models. Inspired by the learning to explain (L2X) paradigm, we propose <jats:sc>L2xGnn<\/jats:sc>, a framework for explainable GNNs which provides <jats:italic>faithful<\/jats:italic> explanations by design. <jats:sc>L2xGnn<\/jats:sc> learns a mechanism for selecting explanatory subgraphs (motifs) which are exclusively used in the GNNs message-passing operations. <jats:sc>L2xGnn<\/jats:sc> is able to select, for each input graph, a subgraph with specific properties such as being sparse and connected. Imposing such constraints on the motifs often leads to more interpretable and effective explanations. Experiments on several datasets suggest that <jats:sc>L2xGnn<\/jats:sc> achieves the same classification accuracy as baseline methods using the entire input graph while ensuring that only the provided explanations are used to make predictions. Moreover, we show that <jats:sc>L2xGnn<\/jats:sc> is able to identify motifs responsible for the graph\u2019s properties it is intended to predict.<\/jats:p>","DOI":"10.1007\/s10994-024-06576-1","type":"journal-article","created":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T17:01:36Z","timestamp":1720803696000},"page":"6787-6809","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["L2XGNN: learning to explain graph neural networks"],"prefix":"10.1007","volume":"113","author":[{"given":"Giuseppe","family":"Serra","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mathias","family":"Niepert","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,7,12]]},"reference":[{"key":"6576_CR1","unstructured":"Agarwal, C., Queen, O., Lakkaraju, H., et\u00a0al. (2022a). An explainable AI library for benchmarking graph explainers. In Workshop on graph learning benchmarks (GLB)."},{"key":"6576_CR2","unstructured":"Agarwal, C., Zitnik, M., & Lakkaraju, H. (2022b). Probing GNN explainers: A rigorous theoretical and empirical analysis of GNN explanation methods. In International conference on artificial intelligence and statistics (pp. 8969\u20138996)."},{"key":"6576_CR3","unstructured":"Baldassarre, F., & Azizpour, H. (2019). Explainability techniques for graph convolutional networks. arXiv preprint arXiv:1905.13686"},{"key":"6576_CR4","doi-asserted-by":"crossref","unstructured":"Borgwardt, K. M., Ong, C. S., Sch\u00f6nauer, S., et al. (2005). Protein function prediction via graph kernels. Bioinformatics,21(suppl_1), i47\u2013i56.","DOI":"10.1093\/bioinformatics\/bti1007"},{"key":"6576_CR5","unstructured":"Chen, J., Song, L., Wainwright, M., et\u00a0al. (2018). Learning to explain: An information-theoretic perspective on model interpretation. In International conference on machine learning, PMLR (pp. 883\u2013892)."},{"issue":"2","key":"6576_CR6","doi-asserted-by":"publisher","first-page":"786","DOI":"10.1021\/jm00106a046","volume":"34","author":"AK Debnath","year":"1991","unstructured":"Debnath, A. K., Lopez de Compadre, R. L., Debnath, G., et al. (1991). Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. Correlation with molecular orbital energies and hydrophobicity. Journal of Medicinal Chemistry, 34(2), 786\u2013797.","journal-title":"Journal of Medicinal Chemistry"},{"issue":"7","key":"6576_CR7","doi-asserted-by":"publisher","first-page":"1895","DOI":"10.1162\/089976698300017197","volume":"10","author":"TG Dietterich","year":"1998","unstructured":"Dietterich, T. G. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7), 1895\u20131923.","journal-title":"Neural Computation"},{"key":"6576_CR8","first-page":"523","volume":"23","author":"J Domke","year":"2010","unstructured":"Domke, J. (2010). Implicit differentiation by perturbation. Advances in Neural Information Processing Systems, 23, 523\u2013531.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6576_CR9","doi-asserted-by":"crossref","unstructured":"Duval, A., & Malliaros, F. D. (2021). Graphsvx: Shapley value explanations for graph neural networks. In Joint European conference on machine learning and knowledge discovery in databases (pp. 302\u2013318). Springer.","DOI":"10.1007\/978-3-030-86520-7_19"},{"key":"6576_CR10","doi-asserted-by":"crossref","unstructured":"Faber, L. K., Moghaddam, A., & Wattenhofer, R. (2021). When comparing to ground truth is wrong: On evaluating GNN explanation methods. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining (pp. 332\u2013341).","DOI":"10.1145\/3447548.3467283"},{"key":"6576_CR11","unstructured":"Feng, Q., Liu, N., Yang, F., et\u00a0al. (2022b). DEGREE: Decomposition based explanation for graph neural networks. In International conference on learning representations."},{"key":"6576_CR12","doi-asserted-by":"crossref","unstructured":"Feng, A., You, C., Wang, S., et\u00a0al. (2022a). Kergnns: Interpretable graph neural networks with graph kernels. In Proceedings of the AAAI conference on artificial intelligence (pp. 6614\u20136622).","DOI":"10.1609\/aaai.v36i6.20615"},{"key":"6576_CR13","unstructured":"Franceschi, L., Niepert, M., Pontil, M., et\u00a0al. (2019). Learning discrete structures for graph neural networks. In International conference on machine learning (pp. 1972\u20131982)."},{"key":"6576_CR14","doi-asserted-by":"crossref","unstructured":"Gao, Y., Sun, T., Bhatt, R., et\u00a0al. (2021). Gnes: Learning to explain graph neural networks. In 2021 IEEE international conference on data mining (ICDM) (pp. 131\u2013140). IEEE.","DOI":"10.1109\/ICDM51629.2021.00023"},{"key":"6576_CR15","unstructured":"Gui, S., Yuan, H., Wang, J., et\u00a0al. (2022). Flowx: Towards explainable graph neural networks via message flows. arXiv preprint arXiv:2206.12987"},{"key":"6576_CR16","unstructured":"Hamilton, W. L., Ying, R., & Leskovec, J. (2017). Inductive representation learning on large graphs. In Proceedings of the 31st international conference on neural information processing systems (pp. 1025\u20131035)."},{"key":"6576_CR17","doi-asserted-by":"crossref","unstructured":"Huang, Q., Yamada, M., Tian, Y., et\u00a0al. (2022). Graphlime: Local interpretable model explanations for graph neural networks. IEEE Transactions on Knowledge and Data Engineering.","DOI":"10.1109\/TKDE.2022.3187455"},{"key":"6576_CR18","unstructured":"Kipf, T. N., & Welling, M. (2017). Semi-supervised classification with graph convolutional networks. In International conference on learning representations."},{"key":"6576_CR19","unstructured":"Lee, J., Lee, I., & Kang, J. (2019). Self-attention graph pooling. In International conference on machine learning, PMLR (pp. 3734\u20133743)."},{"key":"6576_CR20","unstructured":"Lin, W., Lan, H., & Li, B. (2021). Generative causal explanations for graph neural networks. In International conference on machine learning, PMLR (pp. 6666\u20136679)."},{"issue":"240","key":"6576_CR21","first-page":"1","volume":"22","author":"M Liu","year":"2021","unstructured":"Liu, M., Luo, Y., Wang, L., et al. (2021). DIG: A turnkey library for diving into graph deep learning research. Journal of Machine Learning Research, 22(240), 1\u20139.","journal-title":"Journal of Machine Learning Research"},{"key":"6576_CR22","unstructured":"Lucic, A., Ter\u00a0Hoeve, M. A., Tolomei, G., et\u00a0al. (2022). CF-GNNExplainer: Counterfactual explanations for graph neural networks. In International Conference on artificial intelligence and statistics, PMLR (pp. 4499\u20134511)."},{"key":"6576_CR23","first-page":"19620","volume":"33","author":"D Luo","year":"2020","unstructured":"Luo, D., Cheng, W., Xu, D., et al. (2020). Parameterized explainer for graph neural network. Advances in Neural Information Processing Systems, 33, 19620\u201319631.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6576_CR24","unstructured":"Magister, L. C., Kazhdan, D., Singh, V., et\u00a0al. (2021). Gcexplainer: Human-in-the-loop concept-based explanations for graph neural networks. arXiv preprint arXiv:2107.11889"},{"key":"6576_CR25","unstructured":"Miao, S., Liu, M., & Li, P. (2022). Interpretable and generalizable graph learning via stochastic attention mechanism. In International conference on machine learning, PMLR (pp. 15524\u201315543)."},{"key":"6576_CR26","unstructured":"Niepert, M., Minervini, P., & Franceschi, L. (2021). Implicit MLE: Backpropagating through discrete exponential family distributions. In NeurIPS. PMLR, proceedings of machine learning research."},{"key":"6576_CR27","doi-asserted-by":"crossref","unstructured":"Papandreou, G., & Yuille, A. L. (2011). Perturb-and-map random fields: Using discrete optimization to learn and sample from energy models. In 2011 international conference on computer vision (pp. 193\u2013200).","DOI":"10.1109\/ICCV.2011.6126242"},{"key":"6576_CR28","doi-asserted-by":"crossref","unstructured":"Perotti, A., Bajardi, P., Bonchi, F., et\u00a0al. (2023). Explaining identity-aware graph classifiers through the language of motifs. In 2023 International joint conference on neural networks (IJCNN) (pp. 1\u20138). IEEE.","DOI":"10.1109\/IJCNN54540.2023.10191053"},{"key":"6576_CR29","doi-asserted-by":"crossref","unstructured":"Pope, P. E., Kolouri, S., Rostami, M., et\u00a0al. (2019). Explainability methods for graph convolutional neural networks. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 10772\u201310781).","DOI":"10.1109\/CVPR.2019.01103"},{"key":"6576_CR30","first-page":"21030","volume":"35","author":"C Qian","year":"2022","unstructured":"Qian, C., Rattan, G., Geerts, F., et al. (2022). Ordered subgraph aggregation networks. Advances in Neural Information Processing Systems, 35, 21030\u201321045.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6576_CR31","doi-asserted-by":"crossref","unstructured":"Rossi, R., & Ahmed, N. (2015). The network data repository with interactive graph analytics and visualization. In Twenty-ninth AAAI conference on artificial intelligence.","DOI":"10.1609\/aaai.v29i1.9277"},{"key":"6576_CR32","doi-asserted-by":"crossref","unstructured":"Rudin, C. (2018). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. In Proceedings of NeurIPS 2018 workshop on critiquing and correcting trends in learning.","DOI":"10.1038\/s42256-019-0048-x"},{"key":"6576_CR33","first-page":"5898","volume":"33","author":"B Sanchez-Lengeling","year":"2020","unstructured":"Sanchez-Lengeling, B., Wei, J., Lee, B., et al. (2020). Evaluating attribution for graph neural networks. Advances in Neural Information Processing Systems, 33, 5898\u20135910.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6576_CR34","unstructured":"Schlichtkrull, M. S., Cao, N. D., & Titov, I. (2021). Interpreting graph neural networks for NLP with differentiable edge masking. In International conference on learning representations."},{"issue":"11","key":"6576_CR35","doi-asserted-by":"publisher","first-page":"7581","DOI":"10.1109\/TPAMI.2021.3115452","volume":"44","author":"T Schnake","year":"2021","unstructured":"Schnake, T., Eberle, O., Lederer, J., et al. (2021). Higher-order explanations of graph neural networks via relevant walks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11), 7581\u20137596.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"6576_CR36","doi-asserted-by":"crossref","unstructured":"Schwarzenberg, R., H\u00fcbner, M., Harbecke, D., et\u00a0al. (2019). Layerwise relevance visualization in convolutional text graph classifiers. arXiv preprint arXiv:1909.10911","DOI":"10.18653\/v1\/D19-5308"},{"key":"6576_CR37","doi-asserted-by":"crossref","unstructured":"Tan, J., Geng, S., Fu, Z., et al. (2022). Learning and evaluating graph neural network explanations based on counterfactual and factual reasoning. In Proceedings of the ACM web conference, 2022 (pp. 1018\u20131027).","DOI":"10.1145\/3485447.3511948"},{"key":"6576_CR38","unstructured":"Veli\u010dkovi\u0107, P., Cucurull, G., Casanova, A., et\u00a0al. (2018). Graph attention networks. In International conference on learning representations."},{"key":"6576_CR39","first-page":"12225","volume":"33","author":"M Vu","year":"2020","unstructured":"Vu, M., & Thai, M. T. (2020). PGM-explainer: Probabilistic graphical model explanations for graph neural networks. Advances in Neural Information Processing Systems, 33, 12225\u201312235.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6576_CR40","unstructured":"Xu, K., Hu, W., Leskovec, J., et\u00a0al. (2018). How powerful are graph neural networks? In International conference on learning representations."},{"key":"6576_CR41","doi-asserted-by":"crossref","unstructured":"Xuanyuan, H., Barbiero, P., Georgiev, D., et\u00a0al. (2023). Global concept-based interpretability for graph neural networks via neuron analysis. In Proceedings of the AAAI conference on artificial intelligence (pp. 10675\u201310683).","DOI":"10.1609\/aaai.v37i9.26267"},{"key":"6576_CR42","doi-asserted-by":"crossref","unstructured":"Yan, X., Cheng, H., Han, J., et\u00a0al. (2008). Mining significant graph patterns by leap search. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data (pp. 433\u2013444).","DOI":"10.1145\/1376616.1376662"},{"key":"6576_CR43","doi-asserted-by":"crossref","unstructured":"Yanardag, P., & Vishwanathan, S. (2015). Deep graph kernels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 1365\u20131374).","DOI":"10.1145\/2783258.2783417"},{"key":"6576_CR44","first-page":"9240","volume":"32","author":"R Ying","year":"2019","unstructured":"Ying, R., Bourgeois, D., You, J., et al. (2019). Gnnexplainer: Generating explanations for graph neural networks. Advances in Neural Information Processing Systems, 32, 9240.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6576_CR45","first-page":"896","volume":"31","author":"Z Ying","year":"2018","unstructured":"Ying, Z., You, J., Morris, C., et al. (2018). Hierarchical graph representation learning with differentiable pooling. Advances in Neural Information Processing Systems, 31, 896.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6576_CR46","unstructured":"Yu, Z., & Gao, H. (2022). Motifexplainer: A motif-based graph neural network explainer. arXiv preprint arXiv:2202.00519"},{"key":"6576_CR47","unstructured":"Yu, J., Xu, T., Rong, Y., et\u00a0al. (2020). Graph information bottleneck for subgraph recognition. In International conference on learning representations."},{"key":"6576_CR48","doi-asserted-by":"crossref","unstructured":"Yuan, H., Tang, J., Hu, X., et\u00a0al. (2020). XGNN: Towards model-level explanations of graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 430\u2013438).","DOI":"10.1145\/3394486.3403085"},{"key":"6576_CR49","unstructured":"Yuan, H., Yu, H., Wang, J., et\u00a0al. (2021). On explainability of graph neural networks via subgraph explorations. In International conference on machine learning, PMLR (pp. 12241\u201312252)."},{"issue":"5","key":"6576_CR50","first-page":"5782","volume":"45","author":"H Yuan","year":"2022","unstructured":"Yuan, H., Yu, H., Gui, S., et al. (2022). Explainability in graph neural networks: A taxonomic survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5), 5782\u20135799.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"6576_CR51","doi-asserted-by":"crossref","unstructured":"Zhang, M., Cui, Z., Neumann, M., et\u00a0al. (2018). An end-to-end deep learning architecture for graph classification. In Proceedings of the AAAI conference on artificial intelligence.","DOI":"10.1609\/aaai.v32i1.11782"},{"key":"6576_CR52","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Liu, Q., Wang, H., et\u00a0al. (2022). ProtGNN: Towards self-explaining graph neural networks. In Proceedings of the AAAI conference on artificial intelligence (pp. 9127\u20139135).","DOI":"10.1609\/aaai.v36i8.20898"},{"key":"6576_CR53","unstructured":"Zhu, Y., Xu, W., Zhang, J., et\u00a0al. (2021). Deep graph structure learning for robust representations: A survey. arXiv preprint arXiv:2103.03036"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06576-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-024-06576-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06576-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T17:34:49Z","timestamp":1723052089000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-024-06576-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,12]]},"references-count":53,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2024,9]]}},"alternative-id":["6576"],"URL":"https:\/\/doi.org\/10.1007\/s10994-024-06576-1","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,12]]},"assertion":[{"value":"5 December 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 May 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 May 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 July 2024","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}