{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T15:38:50Z","timestamp":1780673930058,"version":"3.54.1"},"reference-count":51,"publisher":"Association for Computing Machinery (ACM)","issue":"5s","license":[{"start":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T00:00:00Z","timestamp":1758844800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"US National Science Foundation","doi-asserted-by":"crossref","award":["CSR-2308530"],"award-info":[{"award-number":["CSR-2308530"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000183","name":"Army Research Office","doi-asserted-by":"crossref","award":["ARO-W911NF-24-1-0240"],"award-info":[{"award-number":["ARO-W911NF-24-1-0240"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Intel CAD SRS Program"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Embed. Comput. Syst."],"published-print":{"date-parts":[[2025,11,30]]},"abstract":"<jats:p>Processing-in-memory (PIM) has been proposed as an alternative computing paradigm for training Deep Neural Networks, including Graph Neural Networks (GNNs). Despite these advancements, training GNN workloads on PIM devices necessitates off-chip memory access.\u00a0This off-chip access is expensive in terms of performance and energy, thereby impacting the overall energy efficiency of the training process on PIM platforms. In this article, we propose a novel hybrid training framework called ERGo that automatically switches from a full-parameter phase to a parameter-efficient phase during the training process with negligible loss in predictive accuracy. The parameter-efficient phase employs low-rank representations of GNN weights, effectively reducing the number of trainable parameters. This helps in reducing the off-chip access during the end-to-end training process on PIM architectures, leading to notable improvements in both performance and energy efficiency. ERGo outperforms the conventional full-parameter GNN training on a PIM-based platform by up to 3.15x in speedup and enhances energy efficiency by 10.5x. Furthermore, ERGo offers additional advantages when utilized on heterogeneous architectures incorporating non-volatile memory (NVM) and static random-access memory (SRAM). Training on the heterogeneous NVM-SRAM architecture typically utilizes NVM for the forward-pass and SRAM for the backward-pass. However, NVM cells suffer from low device endurance and repeated write operations due to weight updates can lead to poor lifetime. ERGo improves the lifetime of NVM devices by freezing the weights and preventing weight updates in the parameter-efficient phase. Experimental results demonstrate that ERGo improves the lifetime of heterogeneous NVM-SRAM architecture by up to 33x in comparison to full-parameter implementation.<\/jats:p>","DOI":"10.1145\/3760402","type":"journal-article","created":{"date-parts":[[2025,8,12]],"date-time":"2025-08-12T11:41:06Z","timestamp":1754998866000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["ERGo: Energy-Efficient Hybrid Graph Neural Network Training on Processing-in-Memory Architectures"],"prefix":"10.1145","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-0422-3365","authenticated-orcid":false,"given":"Pratyush","family":"Dhingra","sequence":"first","affiliation":[{"name":"EECS, Washington State University","place":["Pullman, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-0353-8500","authenticated-orcid":false,"given":"Chibuike","family":"Ugwu","sequence":"additional","affiliation":[{"name":"EECS, Washington State University","place":["Pullman, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3848-5301","authenticated-orcid":false,"given":"Jana","family":"Doppa","sequence":"additional","affiliation":[{"name":"EECS, Washington State University","place":["Pullman, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5930-8531","authenticated-orcid":false,"given":"Partha","family":"Pande","sequence":"additional","affiliation":[{"name":"EECS, Washington State University","place":["Pullman, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,9,26]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"crossref","unstructured":"J. Zhou G. Cui S. Hu Z. Zhang C. Yang Z. Liu L. Wang C. Li and M. Sun. 2020. Graph neural networks: A review of methods and applications. AI Open 1 (2020) 57--81.","DOI":"10.1016\/j.aiopen.2021.01.001"},{"key":"e_1_3_1_3_2","doi-asserted-by":"crossref","unstructured":"Z. Wu S. Pan F. Chen G. Long C. Zhang and P. S. Yu. 2021. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems 32 1 (2021) 4--24.","DOI":"10.1109\/TNNLS.2020.2978386"},{"key":"e_1_3_1_4_2","doi-asserted-by":"crossref","unstructured":"A. Ankit I. El Hajj S. R. Chalamalasetti G. Ndu M. Foltin R. S. Williams J. P. Strachan K. Roy D. Milojicic and W. Hwu. 2020. PANTHER: A programmable architecture for neural network training harnessing energy-efficient ReRAM. IEEE Transactions on Computers 69 8 (2020) 1128--1142.","DOI":"10.1109\/TC.2020.2998456"},{"key":"e_1_3_1_5_2","doi-asserted-by":"crossref","unstructured":"A. I. Arka B. K. Joardar J. R. Doppa and P. P. Pande. 2021. Performance and accuracy tradeoffs for training graph neural networks on ReRAM-based architectures. IEEE Transactions on Computers 70 10 (2021) 1743--1756.","DOI":"10.1109\/TVLSI.2021.3110721"},{"key":"e_1_3_1_6_2","doi-asserted-by":"crossref","unstructured":"Z. He J. Lin R. Ewetz J.-S. Yuan and D. Fan. 2019. Noise injection adaption: End-to-end ReRAM crossbar non-ideal effect adaption for neural network mapping. In Design Automation Conference (DAC).","DOI":"10.1145\/3316781.3317870"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/DAC18072.2020.9218505"},{"key":"e_1_3_1_8_2","article-title":"A practical design-space analysis of compute-in-memory with SRAM","volume":"69","author":"Spetalnick S.","year":"2022","unstructured":"S. Spetalnick and A. Raychowdhury. 2022. A practical design-space analysis of compute-in-memory with SRAM. IEEE Transactions on Circuits and Systems 69, 4 (2022), 1466--1479.","journal-title":"IEEE Transactions on Circuits and Systems"},{"key":"e_1_3_1_9_2","volume-title":"Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","unstructured":"T. N. Sainath, B. Kingsbury, V. Sindhwani, E. Arisoy, and B. Ramabhadran. 2013. Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 6655--6659."},{"key":"e_1_3_1_10_2","volume-title":"Proc. International Conference on Learning Representations (ICLR)","unstructured":"E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. 2022. LoRA: Low-rank adaptation of large language models. In Proc. International Conference on Learning Representations (ICLR). Virtual Event, 2022, arXiv:2106.09685."},{"key":"e_1_3_1_11_2","article-title":"A tutorial on conformal prediction","volume":"9","author":"Shafer G.","year":"2008","unstructured":"G. Shafer and V. Vovk. 2008. A tutorial on conformal prediction. Journal of Machine Learning Research 9 (2008), 371--421.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_1_12_2","volume-title":"Algorithmic Learning in a Random World","author":"Vladimir V.","year":"2005","unstructured":"V. Vladimir, G. Alex, and S. Glenn. 2005. Algorithmic Learning in a Random World. Springer-verlag."},{"key":"e_1_3_1_13_2","doi-asserted-by":"crossref","unstructured":"A. N. Angelopoulos and S. Bates. 2022. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511 (2022).","DOI":"10.1561\/9781638281597"},{"key":"e_1_3_1_14_2","doi-asserted-by":"crossref","unstructured":"M. Yan L. Deng X. Hu L. Liang Y. Feng X. Ye Z. Zhang D. Fan and Y. Xie. 2020. HyGCN: A GCN accelerator with hybrid architecture. In International Symposium on High Performance Computer Architecture (HPCA). 15--29.","DOI":"10.1109\/HPCA47549.2020.00012"},{"key":"e_1_3_1_15_2","doi-asserted-by":"crossref","unstructured":"S. Raman P. Panda A. Sen K. Roy and A. Raghunathan. 2024. NEM-GNN: DAC\/ADC-less scalable reconfigurable graph and sparsity-aware near-memory accelerator for graph neural networks. ACM Transactions on Architecture and Code Optimization 21 3 (2024) 1--25.","DOI":"10.1145\/3652607"},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","unstructured":"R. Sarkar S. Abi-Karam Y. He L. Sathidevi and C. Hao. 2023. FlowGNN: A dataflow architecture for real-time workload-agnostic graph neural network inference. In International Symposium on High Performance Computer Architecture (HPCA). 922--934.","DOI":"10.1109\/HPCA56546.2023.10071015"},{"key":"e_1_3_1_17_2","volume-title":"Computer Society Annual Symposium on VLSI (ISVLSI)","unstructured":"H. Zhong, Y. Zhu, L. Luo, T. Li, C. Wang, Y. Xu, T. Wang, Y. Yu, V. Narayanan, Y. Liu, L. Shi, H. Yang, and X. Li. 2023. Fe-GCN: A 3D FeFET memory based PIM accelerator for graph convolutional networks. In Computer Society Annual Symposium on VLSI (ISVLSI)."},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA53966.2022.00039"},{"key":"e_1_3_1_19_2","volume-title":"Design Automation Conference (DAC'21)","unstructured":"T. Yang, D. Li, Y. Han, Y. Zhao, F. Liu, X. Liang, Z. He, and L. Jiang. 2021. PIMGCN: A ReRAM-based PIM design for graph convolutional network acceleration. In Design Automation Conference (DAC'21)."},{"key":"e_1_3_1_20_2","volume-title":"International Symposium on High Performance Computer Architecture (HPCA'22)","unstructured":"Y. Huang, L. Zheng, P. Yao, Q. Wang, X. Liao, H. Jin, and J. Xue. 2022. Accelerating graph convolutional networks using crossbar-based processing-in-memory architectures. In International Symposium on High Performance Computer Architecture (HPCA'22). 1029--1042."},{"key":"e_1_3_1_21_2","volume-title":"International Conference on Computer-Aided Design (ICCAD'21)","unstructured":"A. I. Arka, B. K. Joardar, J. R. Doppa, P. P. Pande, and K. Chakrabarty. 2021. DARe: DropLayer-aware manycore ReRAM architecture for training graph neural networks. In International Conference on Computer-Aided Design (ICCAD'21)."},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2017.55"},{"key":"e_1_3_1_23_2","doi-asserted-by":"crossref","unstructured":"Y. Ma J. Lu J. Zhao Y. Wang H. Li and X. Li. 2024. DCIM-GCN: Digital computing-in-memory accelerator for graph convolutional network. IEEE Transactions on Circuits and Systems 71 1 (2024) 233--237.","DOI":"10.1109\/TCSI.2024.3384748"},{"key":"e_1_3_1_24_2","doi-asserted-by":"crossref","unstructured":"X. Peng S. Huang Y. Luo X. Sun and S. Yu. 2021. DNN+NeuroSim V2.0: An end-to-end benchmarking framework for compute-in-memory accelerators for on-chip training. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 40 11 (2021) 2306--2319.","DOI":"10.1109\/TCAD.2020.3043731"},{"key":"e_1_3_1_25_2","doi-asserted-by":"crossref","unstructured":"H. Zhang J. Liu J. Bai K. Wang C. Xu N. Xu W. Wang and W. Zhao. 2022. HD-CIM: Hybrid-device computing-in-memory structure based on MRAM and SRAM to reduce weight loading energy of neural networks. IEEE Transactions on Circuits and Systems 69 4 (2022) 1770--1774.","DOI":"10.1109\/TCSI.2022.3199440"},{"key":"e_1_3_1_26_2","doi-asserted-by":"crossref","unstructured":"G. Krishnan Z. Wang I. Yeo L. Yang J. Meng M. Liehr R. V. Joshi N. C. Cady D. Fan J. Seo and Y. Cao. 2022. Hybrid RRAM\/SRAM in-memory computing for robust DNN acceleration. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 41 11 (2022) 4595--4608.","DOI":"10.1109\/TCAD.2022.3197516"},{"key":"e_1_3_1_27_2","volume-title":"International Symposium on Computer Architecture (ISCA)","unstructured":"S. Resch, S. K. Khatamifard, Z. Surucu, M. Luo, A. Cperformato, and U. R. Karpuzcu. 2023. On endurance of processing in (Nonvolatile) memory. In International Symposium on Computer Architecture (ISCA)."},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/JETCAS.2021.3127517"},{"key":"e_1_3_1_29_2","doi-asserted-by":"crossref","unstructured":"L. Xia B. Li T. Tang P. Gu P.-Y. Chen S. Yu Y. Cao Y. Wang Y. Xie and H. Yang. 2018. Stuck-at fault tolerance in RRAM computing systems. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 8 1 (2018) 102--115.","DOI":"10.1109\/JETCAS.2017.2776980"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2018.2883981"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.3850\/9783981537079_0032"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/1669112.1669157"},{"key":"e_1_3_1_33_2","doi-asserted-by":"crossref","unstructured":"L. Xia M. Liu X. Ning K. Chakrabarty and Y. Wang. 2019. Fault-tolerant training enabled by on-line fault detection for RRAM-based neural computing systems. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 38 9 (2019) 1611--1624.","DOI":"10.1109\/TCAD.2018.2855145"},{"key":"e_1_3_1_34_2","volume-title":"Design, Automation & Test in Europe Conference & Exhibition (DATE)","unstructured":"P. Dhingra, C. Ogbogu, B. K. Joardar, J. R. Doppa, A. Kalyanaraman, and P. P. Pande. 2024. FARe: Fault-aware GNN training on ReRAM-based PIM accelerators. In Design, Automation & Test in Europe Conference & Exhibition (DATE)"},{"key":"e_1_3_1_35_2","doi-asserted-by":"crossref","unstructured":"X. Yang H. Yang J. R. Doppa P. P. Pande K. Chakrabarty and H. Li. 2023. ESSENCE: Exploiting structured stochastic gradient pruning for endurance-aware ReRAM-based in-memory training systems. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 42 7 (2023) 2187--2199.","DOI":"10.1109\/TCAD.2022.3216546"},{"key":"e_1_3_1_36_2","volume-title":"Design Automation Conference (DAC'18)","unstructured":"Y. Cai, Y. Lin, L. Xia, X. Chen, S. Han, Y. Wang, and H. Yang. 2018. Long live TIME: Improving lifetime for training-in-memory engines by structured gradient sparsification. In Design Automation Conference (DAC'18)."},{"key":"e_1_3_1_37_2","doi-asserted-by":"crossref","unstructured":"A. Shafiee A. Nag N. Muralimanohar R. Balasubramonian J. P. Strachan M. Hu R. S. Williams and V. Srikumar. 2016. ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars. In International Symposium on Computer Architecture (ISCA'16).","DOI":"10.1109\/ISCA.2016.12"},{"key":"e_1_3_1_38_2","unstructured":"A. Aghajanyan L. Zettlemoyer and S. Gupta. 2020. Intrinsic dimensionality explains the effectiveness of language model finetuning. In arXiv preprint arXiv:2012.13255 (2020)."},{"key":"e_1_3_1_39_2","article-title":"Conformalized quantile regression","author":"Romano Y.","year":"2019","unstructured":"Y. Romano, E. Patterson, and E. J. Cand\u00e8s. 2019. Conformalized quantile regression. In Advances in Neural Information Processing Systems 32 (2019), 3543--3553.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_40_2","unstructured":"W. Hu M. Fey M. Zitnik Y. Dong H. Ren B. Liu M. Catasta and J. Leskovec. 2020. Open graph benchmark: Datasets for machine learning on graphs. In Advances in Neural Information Processing Systems (NeurIPS) 33 (2020) 22118--22133."},{"key":"e_1_3_1_41_2","unstructured":"T. N. Kipf and M. Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proc. International Conference on Learning Representations (ICLR'17). arXiv:1609.02907."},{"key":"e_1_3_1_42_2","unstructured":"P. Veli\u010dkovi\u0107 G. Cucurull A. Casanova A. Romero P. Li\u00c32 and Y. Bengio. 2018. Graph attention networks. In Proc. International Conference on Learning Representations (ICLR'18)."},{"key":"e_1_3_1_43_2","article-title":"Inductive representation learning on large graphs","volume":"30","author":"Hamilton W. L.","year":"2017","unstructured":"W. L. Hamilton, R. Ying, and J. Leskovec. 2017. Inductive representation learning on large graphs. NeurIPS 30 (2017), 1024--1034.","journal-title":"NeurIPS"},{"issue":"6","key":"e_1_3_1_44_2","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1137\/S1064827595287997","article-title":"A fast and high quality multilevel scheme for partitioning irregular graphs","volume":"20","author":"Karypis G.","year":"1998","unstructured":"G. Karypis and V. Kumar. 1998. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on Scientific Computing 20, 6 (1998), 359\u2013392.","journal-title":"SIAM Journal on Scientific Computing"},{"key":"e_1_3_1_45_2","unstructured":"JESD235D JEDEC Standard \u201cHigh Bandwidth Memory DRAM (HBM1 HBM2) \u201d JEDEC SOLID STATE TECHNOLOGY ASSOCIATION USA 2021."},{"key":"e_1_3_1_46_2","unstructured":"D. P. Kingma and J. Ba. 2015. Adam: A method for stochastic optimization. In Proc. International Conference on Learning Representations (ICLR'15)."},{"key":"e_1_3_1_47_2","unstructured":"S. Ruder. 2016. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747."},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2025.3531255"},{"key":"e_1_3_1_49_2","doi-asserted-by":"crossref","unstructured":"R. Agarwal P. Cheng P. Shah B. Wilkerson R. Swaminathan J. Wuu and C. Mandalapu. 2022. 3D packaging for heterogeneous integration. Electronic Components and Technology Conference (ECTC'22). 1103--1107.","DOI":"10.1109\/ECTC51906.2022.00178"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ECTC.2019.00095"},{"key":"e_1_3_1_51_2","doi-asserted-by":"crossref","unstructured":"S. V. Huylenbroeck M. Stucchi Y. Li J. Slabbekoorn N. Tutunjyan S. S. N. Jourdan L. Bogaerts F. Beirnaert G. Beyer and E. Beyne. 2016. Small pitch high aspect ratio via-last TSV module. Electronic Components and Technology Conference (ECTC). 43--49.","DOI":"10.1109\/ECTC.2016.155"},{"key":"e_1_3_1_52_2","doi-asserted-by":"crossref","unstructured":"C.-C. Lee C. Hung C. Cheung P.-F. Yang C.-L. Kao; D.-L. Chen M. Shih C.-L. C. Chien Y.-H. Hsiao L.-C. Chen M. Su M. Alfano J. Siegel J. Din and B. Black. 2016. An overview of the development of a GPU with integrated HBM on silicon interposer. Electronic Components and Technology Conference (ECTC'16). 1439--1444.","DOI":"10.1109\/ECTC.2016.348"}],"container-title":["ACM Transactions on Embedded Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3760402","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3760402","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,1]],"date-time":"2025-10-01T17:13:02Z","timestamp":1759338782000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3760402"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,26]]},"references-count":51,"journal-issue":{"issue":"5s","published-print":{"date-parts":[[2025,11,30]]}},"alternative-id":["10.1145\/3760402"],"URL":"https:\/\/doi.org\/10.1145\/3760402","relation":{},"ISSN":["1539-9087","1558-3465"],"issn-type":[{"value":"1539-9087","type":"print"},{"value":"1558-3465","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,26]]},"assertion":[{"value":"2025-08-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-07","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-09-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}