{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T10:50:23Z","timestamp":1770979823670,"version":"3.50.1"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"OOPSLA2","license":[{"start":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T00:00:00Z","timestamp":1759968000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["CCF-2316233"],"award-info":[{"award-number":["CCF-2316233"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000028","name":"Semiconductor Research Corporation","doi-asserted-by":"publisher","award":["JUMP 2.0, ACE Center"],"award-info":[{"award-number":["JUMP 2.0, ACE Center"]}],"id":[{"id":"10.13039\/100000028","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2025,10,9]]},"abstract":"<jats:p>Multiple frameworks and optimizations have been proposed for accelerating Graph Neural Network (GNN) workloads over the years, achieving sizable runtime performance improvements. However, we notice that existing systems usually explore optimizing either at the intra-operator level or at the inter-operator level, missing synergies that exist due to their compositions. Further, most existing works focus primarily on optimizing the forward computation of GNNs, often overlooking opportunities for training-specific optimizations.  \nTo exploit these missed optimization opportunities, we introduce GALA, a domain-specific language (DSL) and a compiler that allows composing optimizations at different levels. The GALA DSL exposes intra-operator transformations as scheduling commands, while we introduce novel inter-operator transformations as part of the compiler. The composition of these transformations is made possible through the introduction of two novel intermediate representations (IR) in the GALA compiler that tracks and composes transformations at both the intra- and inter-operator levels. Further, the IRs maintain a global view of the GNN program, including its training process. This allows us to introduce training-specific transformations to aggressively optimize GNN training. Our evaluations show that GALA achieves a geo-mean speedup of 2.55\u00d7 for inference and 2.52\u00d7 for training across multiple systems, graphs, and GNN models. We also show that GALA performs well across different graph sizes and GNN model configurations, as well as allows users to explore different methods of performing similar optimizations leading to different tradeoff spaces.<\/jats:p>","DOI":"10.1145\/3763113","type":"journal-article","created":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T08:49:50Z","timestamp":1759999790000},"page":"1754-1782","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["GALA: A High Performance Graph Neural Network Acceleration LAnguage and Compiler"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9804-3994","authenticated-orcid":false,"given":"Damitha","family":"Lenadora","sequence":"first","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-0826-3553","authenticated-orcid":false,"given":"Nikhil","family":"Jayakumar","sequence":"additional","affiliation":[{"name":"University of Texas at Austin, Austin, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2161-2212","authenticated-orcid":false,"given":"Chamika","family":"Sudusinghe","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8140-2321","authenticated-orcid":false,"given":"Charith","family":"Mendis","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,10,9]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Abien Fred Agarap. 2018. Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3579990.3580020"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3519939.3523442"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3620665.3640366"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3591236"},{"key":"e_1_2_2_6_1","unstructured":"Filippo Maria Bianchi Daniele Grattarola and Cesare Alippi. 2020. Spectral Clustering with Graph Neural Networks for Graph Pooling. arxiv:1907.00481. arxiv:1907.00481"},{"key":"e_1_2_2_7_1","doi-asserted-by":"crossref","unstructured":"Pasquale Bove Alessio Micheli Paolo Milazzo Marco Podda et al. 2020. Prediction of Dynamical Properties of Biochemical Pathways with Graph Neural Networks.. In Bioinformatics. 32\u201343.","DOI":"10.5220\/0008964700320043"},{"key":"e_1_2_2_8_1","unstructured":"Shaked Brody Uri Alon and Eran Yahav. 2022. How Attentive are Graph Attention Networks? arxiv:2105.14491."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3620665.3640399"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/3571885.3571927"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126908.3126936"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3581784.3607097"},{"key":"e_1_2_2_13_1","unstructured":"Adhitha Dias Logan Anderson Kirshanthan Sundararajah Artem Pelenitsyn and Milind Kulkarni. 2024. SparseAuto: An Auto-Scheduler for Sparse Tensor Computations Using Recursive Loop Nest Restructuring. arxiv:2311.09549. arxiv:2311.09549"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2024.126170"},{"key":"e_1_2_2_15_1","volume-title":"Handbook of Weighted Automata","author":"Droste Manfred","year":"2014","unstructured":"Manfred Droste, Werner Kuich, and Heiko Vogler. 2009. Handbook of Weighted Automata (1st ed.). Springer Publishing Company, Incorporated. isbn:3642014917","edition":"1"},{"key":"e_1_2_2_16_1","unstructured":"Jian Du Shanghang Zhang Guanhang Wu Jose M. F. Moura and Soummya Kar. 2018. Topology Adaptive Graph Convolutional Networks. arxiv:1710.10370."},{"key":"e_1_2_2_17_1","volume-title":"Machine Learning for Astrophysics","author":"Farsian Farida","unstructured":"Farida Farsian, Federico Marulli, Lauro Moscardini, and Carlo Giocoli. 2023. New Applications of Graph Neural Networks in Cosmology. In Machine Learning for Astrophysics, Filomena Bufano, Simone Riggi, Eva Sciacca, and Francesco Schilliro (Eds.). Springer International Publishing, Cham. 35\u201338. isbn:978-3-031-34167-0"},{"key":"e_1_2_2_18_1","volume-title":"Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds.","author":"Fey Matthias","unstructured":"Matthias Fey and Jan E. Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3470496.3527403"},{"key":"e_1_2_2_20_1","unstructured":"Will Hamilton Zhitao Ying and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in neural information processing systems. 1024\u20131034."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.renene.2025.122520"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3293883.3295712"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00167"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3627703.3650063"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3437801.3441585"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/PACT58117.2023.00020"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626183.3659985"},{"key":"e_1_2_2_28_1","volume-title":"Proceedings of Machine Learning and Systems, A. Smola, A. Dimakis, and I. Stoica (Eds.). 3, 387\u2013400","author":"Kaufman Sam","year":"2021","unstructured":"Sam Kaufman, Phitchaya Phothilimthana, Yanqi Zhou, Charith Mendis, Sudip Roy, Amit Sabne, and Mike Burrows. 2021. A Learned Performance Model for Tensor Processing Units. In Proceedings of Machine Learning and Systems, A. Smola, A. Dimakis, and I. Stoica (Eds.). 3, 387\u2013400. https:\/\/proceedings.mlsys.org\/paper\/2021\/file\/85d8ce590ad8981ca2c8286f79f59954-Paper.pdf"},{"key":"e_1_2_2_29_1","volume-title":"Kipf and Max Welling","author":"Thomas","year":"2016","unstructured":"Thomas N. Kipf and Max Welling. 2016. Semi-Supervised Classification with Graph Convolutional Networks. CoRR, abs\/1609.02907 (2016), arXiv:1609.02907. arxiv:1609.02907"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133901"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3656445"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.16923829"},{"key":"e_1_2_2_33_1","volume-title":"John Levine and Levine John. 2009. Flex & Bison","unstructured":"John Levine and Levine John. 2009. Flex & Bison (1st ed.). O\u2019Reilly Media, Inc.. isbn:0596155972","edition":"1"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jobe.2025.112048"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201383"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00138-024-01656-7"},{"key":"e_1_2_2_37_1","volume-title":"GPU Technology Conference. 12","author":"Naumov Maxim","year":"2010","unstructured":"Maxim Naumov, L Chien, Philippe Vandermersch, and Ujval Kapasi. 2010. Cusparse library. In GPU Technology Conference. 12."},{"key":"e_1_2_2_38_1","unstructured":"Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in PyTorch."},{"key":"e_1_2_2_39_1","volume-title":"Languages and Compilers for Parallel Computing","author":"Qiu Shenghao","unstructured":"Shenghao Qiu, Liang You, and Zheng Wang. 2022. Optimizing Sparse Matrix Multiplications for Graph Neural Networks. In Languages and Compilers for Parallel Computing, Xiaoming Li and Sunita Chandrasekaran (Eds.). Springer International Publishing, Cham. 101\u2013117. isbn:978-3-030-99372-6"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2499370.2462176"},{"key":"e_1_2_2_41_1","volume-title":"FusedMM: A Unified SDDMM-SpMM Kernel for Graph Embedding and Graph Neural Networks. In 35th Proceedings of IEEE IPDPS.","author":"Rahman Md","year":"2021","unstructured":"Md Rahman, Majedul Haque Sujon, Ariful Azad, et al. 2021. FusedMM: A Unified SDDMM-SpMM Kernel for Graph Embedding and Graph Neural Networks. In 35th Proceedings of IEEE IPDPS."},{"key":"e_1_2_2_42_1","volume-title":"XLA : Compiling Machine Learning for Peak Performance.","author":"Sabne Amit","year":"2020","unstructured":"Amit Sabne. 2020. XLA : Compiling Machine Learning for Peak Performance."},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3583780.3615091"},{"key":"e_1_2_2_44_1","volume-title":"Graph Attention Networks. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=rJXMpikCZ","author":"Veli\u010dkovi\u0107 Petar","year":"2018","unstructured":"Petar Veli\u010dkovi\u0107, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Li\u00f2, and Yoshua Bengio. 2018. Graph Attention Networks. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=rJXMpikCZ"},{"key":"e_1_2_2_45_1","unstructured":"Minjie Wang Lingfan Yu Da Zheng Quan Gan Yu Gai Zihao Ye Mufei Li Jinjing Zhou Qi Huang Chao Ma Ziyue Huang Qipeng Guo Hao Zhang Haibin Lin Junbo Zhao Jinyang Li Alexander Smola and Zheng Zhang. 2019. Deep Graph Library: Towards Efficient and Scalable Deep Learning on Graphs. arxiv:1909.01315."},{"key":"e_1_2_2_46_1","unstructured":"Wenxi Wang Yang Hu Mohit Tiwari Sarfraz Khurshid Kenneth McMillan and Risto Miikkulainen. 2024. NeuroBack: Improving CDCL SAT Solving using Graph Neural Networks. arxiv:2110.14053. arxiv:2110.14053"},{"key":"e_1_2_2_47_1","volume-title":"GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21)","author":"Wang Yuke","year":"2021","unstructured":"Yuke Wang, Boyuan Feng, Gushu Li, Shuangchen Li, Lei Deng, Yuan Xie, and Yufei Ding. 2021. GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21). USENIX Association, 515\u2013531. isbn:978-1-939133-22-9 https:\/\/www.usenix.org\/conference\/osdi21\/presentation\/wang-yuke"},{"key":"e_1_2_2_48_1","volume-title":"Christopher Fifty, Tao Yu, and Kilian Q. Weinberger.","author":"Wu Felix","year":"2019","unstructured":"Felix Wu, Tianyi Zhang, Amauri Holanda de Souza Jr. au2, Christopher Fifty, Tao Yu, and Kilian Q. Weinberger. 2019. Simplifying Graph Convolutional Networks. arxiv:1902.07153."},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3620666.3651322"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447786.3456247"},{"key":"e_1_2_2_51_1","volume-title":"Proceedings of Machine Learning and Systems, D. Marculescu, Y. Chi, and C. Wu (Eds.). 4, 515\u2013528","author":"Xie Zhiqiang","year":"2022","unstructured":"Zhiqiang Xie, Minjie Wang, Zihao Ye, Zheng Zhang, and Rui Fan. 2022. Graphiler: Optimizing Graph Neural Networks with Message Passing Data Flow Graph. In Proceedings of Machine Learning and Systems, D. Marculescu, Y. Chi, and C. Wu (Eds.). 4, 515\u2013528. https:\/\/proceedings.mlsys.org\/paper\/2022\/file\/a87ff679a2f3e71d9181a67b7542122c-Paper.pdf"},{"key":"e_1_2_2_52_1","volume-title":"International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=ryGs6iA5Km","author":"Xu Keyulu","year":"2019","unstructured":"Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks? In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=ryGs6iA5Km"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/LCA.2020.2970395"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3582016.3582047"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3572848.3577506"},{"key":"e_1_2_2_56_1","volume-title":"IO, and Memory Perspective. 4","author":"Zhang Hengrui","year":"2022","unstructured":"Hengrui Zhang, Zhongming Yu, Guohao Dai, Guyue Huang, Yufei Ding, Yuan Xie, and Yu Wang. 2022. Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective. 4 (2022), 467\u2013484. https:\/\/proceedings.mlsys.org\/paper\/2022\/file\/9a1158154dfa42caddbd0694a4e9bdc8-Paper.pdf"},{"key":"e_1_2_2_57_1","unstructured":"Muhan Zhang and Yixin Chen. 2018. Link prediction based on graph neural networks. In Advances in Neural Information Processing Systems. 5165\u20135175."},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3276491"},{"key":"e_1_2_2_59_1","unstructured":"Kaixiong Zhou Qingquan Song Xiao Huang and Xia Hu. 2019. Auto-GNN: Neural Architecture Search of Graph Neural Networks. arxiv:1909.03184. arxiv:1909.03184"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3575693.3575723"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3763113","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3763113","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:47:54Z","timestamp":1760032074000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3763113"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,9]]},"references-count":60,"journal-issue":{"issue":"OOPSLA2","published-print":{"date-parts":[[2025,10,9]]}},"alternative-id":["10.1145\/3763113"],"URL":"https:\/\/doi.org\/10.1145\/3763113","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,9]]},"assertion":[{"value":"2025-03-25","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-12","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}