{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T14:37:05Z","timestamp":1787236625054,"version":"build-2736575974"},"reference-count":33,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"1","funder":[{"name":"Exascale Computing Project"},{"name":"MMICC","award":["SEA-CROGS"],"award-info":[{"award-number":["SEA-CROGS"]}]},{"DOI":"10.13039\/100006132","name":"Office of Science","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006132","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM Rev."],"published-print":{"date-parts":[[2025,3,31]]},"abstract":"<jats:p>Abstract.<\/jats:p>\n                  <jats:p>Sparse matrix computations are ubiquitous in scientific computing. Given the recent interest in scientific machine learning, it is natural to ask how sparse matrix computations can leverage neural networks (NNs). Unfortunately, multilayer perceptron (MLP) NNs are typically not natural for either graph or sparse matrix computations. The issue lies with the fact that MLPs require fixed-sized inputs, while scientific applications generally generate sparse matrices with arbitrary dimensions and a wide range of different nonzero patterns (or matrix graph vertex interconnections). While convolutional NNs could possibly address matrix graphs where all vertices have the same number of nearest neighbors, a more general approach is needed for arbitrary sparse matrices, e.g., those arising from discretized partial differential equations on unstructured meshes. Graph neural networks (GNNs) are one such approach suitable to sparse matrices. The key idea is to define aggregation functions (e.g., summations) that operate on variable-size input data to produce data of a fixed output size so that MLPs can be applied. The goal of this paper is to provide an introduction to GNNs for a numerical linear algebra audience. Concrete GNN examples are provided to illustrate how many common linear algebra tasks can be accomplished using GNNs. We focus on iterative and multigrid methods that employ computational kernels such as matrix-vector products, interpolation, relaxation methods, and strength-of-connection measures. Our GNN examples include cases where parameters are determined a priori as well as cases where parameters must be learned. The intent of this paper is to help computational scientists understand how GNNs can be used to adapt machine learning concepts to computational tasks associated with sparse matrices. It is hoped that this understanding will further stimulate data-driven extensions of classical sparse linear algebra tasks.<\/jats:p>","DOI":"10.1137\/23m1609786","type":"journal-article","created":{"date-parts":[[2025,2,6]],"date-time":"2025-02-06T13:00:56Z","timestamp":1738846856000},"page":"141-175","source":"Crossref","is-referenced-by-count":8,"title":["Graph Neural Networks and Applied Linear Algebra"],"prefix":"10.1137","volume":"67","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5224-9113","authenticated-orcid":true,"given":"Nicholas S.","family":"Moore","sequence":"first","affiliation":[{"name":"College of Engineering, West Texas A&M University, Canyon, TX 79016 USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eric C.","family":"Cyr","sequence":"additional","affiliation":[{"name":"Sandia National Laboratories, Albuquerque, NM 87185 USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4803-4416","authenticated-orcid":true,"given":"Peter","family":"Ohm","sequence":"additional","affiliation":[{"name":"Complex Phenomena Unified Simulation Research Team, RIKEN Center for Computational Science, Chuo-ku, Kobe, Hyogo, 650-0047, Japan."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher M.","family":"Siefert","sequence":"additional","affiliation":[{"name":"Computational Shock and Multiphysics, Sandia National Laboratories, Albuquerque, NM 87185 USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-6951-973X","authenticated-orcid":true,"given":"Raymond S.","family":"Tuminaro","sequence":"additional","affiliation":[{"name":"Computational Science and Mathematics Department, Sandia National Laboratories, Livermore, CA 94551 USA."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2025,2,6]]},"reference":[{"key":"ref1","volume-title":"TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems","author":"Abadi M.","year":"2015"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.2172\/1577437"},{"key":"ref3","unstructured":"P. W. Battaglia, J. B. Hamrick, V. Bapst, A. Sanchez-Gonzalez, V. Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner, C. Gulcehre, F. Song, A. Ballard, J. Gilmer, G. Dahl, A. Vaswani, K. Allen, C. Nash, V. Langston, C. Dyer, N. Heess, D. Wierstra, P. Kohli, M. Botvinick, O. Vinyals, Y. Li, and R. Pascanu, Relational Inductive Biases, Deep Learning, and Graph Networks, http:\/\/arxiv.org\/abs\/1806.01261, 2018."},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.1137\/16M1080173"},{"key":"ref5","volume-title":"JAX: Composable Transformations of Python+NumPy Programs","author":"Bradbury J.","year":"2018"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898719505"},{"key":"ref7","first-page":"512","volume-title":"Mathematical and Scientific Machine Learning","author":"Cyr E. C.","year":"2020"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1002\/nla.559"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2018.01.007"},{"key":"ref10","unstructured":"B. Edwards, Hear Elvis Sing Baby Got Back using AI\u2014and Learn How It Was Made, https:\/\/arstechnica.com\/information-technology\/2023\/08\/hear-elvis-sing-baby-got-back-using-ai-and-learn-how-it-was-made\/ (accessed 2024-10-19)."},{"key":"ref11","volume-title":"ICLR Workshop on Representation Learning on Graphs and Manifolds","author":"Fey M.","year":"2019"},{"key":"ref12","unstructured":"J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, Neural Message Passing for Quantum Chemistry, http:\/\/arxiv.org\/abs\/1704.01212, 2017."},{"key":"ref13","unstructured":"X. Glorot and Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, 2010, pp. 249\u2013256."},{"key":"ref14","volume-title":"Deep Learning","author":"Goodfellow I.","year":"2016"},{"key":"ref15","doi-asserted-by":"crossref","unstructured":"M. Gori, G. Monfardini, and F. Scarselli, A new model for learning in graph domains, in Proceedings of the 2005 IEEE International Joint Conference on Neural Networks, Vol. 2, 2005, pp. 729\u2013734, https:\/\/doi.org\/10.1109\/IJCNN.2005.1555942.","DOI":"10.1109\/IJCNN.2005.1555942"},{"key":"ref16","doi-asserted-by":"crossref","unstructured":"K. He, X. Zhang, S. Ren, and J. Sun, Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification, in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1026\u20131034.","DOI":"10.1109\/ICCV.2015.123"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1145\/1089014.1089021"},{"key":"ref18","unstructured":"Hugging Face Transformers, https:\/\/huggingface.co\/docs\/transformers\/index (accessed 2024-10-19)."},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.3389\/fdata.2022.1001063"},{"key":"ref20","unstructured":"D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, https:\/\/arxiv.org\/abs\/1412.6980, 2014."},{"key":"ref21","unstructured":"I. Luz, M. Galun, H. Maron, R. Basri, and I. Yavneh, Learning algebraic multigrid using graph neural networks, in Proceedings of the 37th International Conference on Machine Learning,  H. D. III and A. Singh, eds. Proc. Mach. Learn. Res. 119, PMLR, 2020, pp. 6489\u20136499, https:\/\/proceedings.mlr.press\/v119\/luz20a.html."},{"key":"ref22","doi-asserted-by":"crossref","unstructured":"N. S. Moore, E. C. Cyr, and C. M. Siefert, Learning an Algebriac Multrigrid Interpolation Operator using a Modified Graphnet Architecture, Tech.\u00a0report, Sandia National Laboratories, 2021, https:\/\/doi.org\/10.2172\/1859673.","DOI":"10.2172\/1859673"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1007\/b98874"},{"key":"ref24","first-page":"8024","volume-title":"Advances in Neural Information Processing Systems 32","author":"Paszke A.","year":"2019"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729586"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898718003"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2008.2005605"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1017\/dce.2022.24"},{"key":"ref29","first-page":"12129","volume-title":"Advances in Neural Information Processing Systems","volume":"34","author":"Taghibakhshi A.","year":"2021"},{"key":"ref30","first-page":"7222","volume-title":"Advances in Neural Information Processing Systems","volume":"35","author":"Taghibakhshi A.","year":"2022"},{"key":"ref31","unstructured":"A. Taghibakhshi, N. Nytko, T. U. Zaman, S. MacLachlan, L. Olson, and M. West, MG-GNN: Multigrid graph neural networks for learning multilevel domain decomposition methods, in Proceedings of the 40th International Conference on Machine Learning, ICML\u201923, JMLR.org, 2023."},{"key":"ref32","first-page":"26","volume":"4","author":"Tieleman T.","year":"2012","journal-title":"COURSERA Neural Networks for Machine Learning"},{"key":"ref33","volume-title":"Advances in Neural Information Processing Systems","volume":"32","author":"Wang W.","year":"2019"}],"container-title":["SIAM Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/23M1609786","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T13:39:34Z","timestamp":1787233174000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/23M1609786"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,6]]},"references-count":33,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,3,31]]}},"alternative-id":["10.1137\/23M1609786"],"URL":"https:\/\/doi.org\/10.1137\/23m1609786","relation":{},"ISSN":["0036-1445","1095-7200"],"issn-type":[{"value":"0036-1445","type":"print"},{"value":"1095-7200","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,6]]}}}