{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T05:31:54Z","timestamp":1780723914388,"version":"3.54.1"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"name":"Joint Funds of the National Key Research and Development Program of China","award":["2020YFB1406902"],"award-info":[{"award-number":["2020YFB1406902"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["U22A2036, 62202123"],"award-info":[{"award-number":["U22A2036, 62202123"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies","award":["2022B1212010005"],"award-info":[{"award-number":["2022B1212010005"]}]},{"DOI":"10.13039\/501100005046","name":"Natural Science Foundation of Heilongjiang Province","doi-asserted-by":"crossref","award":["LH2024F022"],"award-info":[{"award-number":["LH2024F022"]}],"id":[{"id":"10.13039\/501100005046","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:p>Homomorphic encryption (HE) represents an encryption technology that allows for direct computation on encrypted data without requiring decryption. However, the substantial computational complexity and significant latency associated with HE has impeded its broader adoption in practical applications. To address these challenges, we propose a GPU-based acceleration framework, namely HEngine, tailored for homomorphic encryption tasks. Specifically, we first propose a warp shuffle-based optimization method for two key phases, i.e., inverse Chinese Remainder Theorem (ICRT) and number theoretic transformation (NTT), to mitigate synchronization overhead in homomorphic encryption. Secondly, we propose to fuse the NTT kernel with the inner product kernel to address the imbalance between memory access and computation. Thirdly, considering the potential difference in the amount of tasks of users in the real-world, we design two different encoding methods for small batch and large batch inference tasks to improve computational efficiency. Finally, experiments demonstrate that our proposed framework achieves a 218\u00d7 speedup on homomorphic multiplication tasks compared with the CPU-based SEAL library. In addition, for convolutional neural network inference tasks on shallow network structures, our proposed framework achieves amortized inference performance at the millisecond level and sub-millisecond level on small batch and large batch data, respectively. For convolutional neural network inference tasks on deeper network structures (i.e., ResNet-20), our proposed framework achieves second-level inference.<\/jats:p>","DOI":"10.1145\/3732942","type":"journal-article","created":{"date-parts":[[2025,4,28]],"date-time":"2025-04-28T11:07:07Z","timestamp":1745838427000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["HEngine: A High Performance Optimization Framework on a GPU for Homomorphic Encryption"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-3638-3656","authenticated-orcid":false,"given":"Jinghao","family":"Zhao","sequence":"first","affiliation":[{"name":"Harbin Institute of Technology","place":["Harbin, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8386-0131","authenticated-orcid":false,"given":"Hongwei","family":"Yang","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology","place":["Harbin, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0043-4370","authenticated-orcid":false,"given":"Meng","family":"Hao","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology","place":["Harbin, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4783-876X","authenticated-orcid":false,"given":"Weizhe","family":"Zhang","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology Shenzhen","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6494-775X","authenticated-orcid":false,"given":"Hui","family":"He","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology","place":["Harbin, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7502-7094","authenticated-orcid":false,"given":"Desheng","family":"Wang","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,7,2]]},"reference":[{"issue":"2","key":"e_1_3_2_2_2","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1109\/TASSP.1974.1162555","article-title":"Fast convolution using fermat number transforms with applications to digital filtering","volume":"22","author":"Agarwal RC","year":"1974","unstructured":"RC Agarwal and C Burrus. 1974. Fast convolution using fermat number transforms with applications to digital filtering. IEEE Transactions on Acoustics, Speech, and Signal Processing 22, 2 (1974), 87\u201397.","journal-title":"IEEE Transactions on Acoustics, Speech, and Signal Processing"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/TETC.2020.3014636"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/SOLI.2018.8476725"},{"issue":"2","key":"e_1_3_2_5_2","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1109\/TPDS.2020.3021238","article-title":"Multi-GPU design and performance evaluation of homomorphic encryption on GPU clusters","volume":"32","author":"Badawi Ahmad Al","year":"2020","unstructured":"Ahmad Al Badawi, Bharadwaj Veeravalli, Jie Lin, Nan Xiao, Matsumura Kazuaki, and Aung Khin Mi Mi. 2020. Multi-GPU design and performance evaluation of homomorphic encryption on GPU clusters. IEEE Transactions on Parallel and Distributed Systems 32, 2 (2020), 379\u2013391.","journal-title":"IEEE Transactions on Parallel and Distributed Systems"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","unstructured":"Ahmad Al Badawi Bharadwaj Veeravalli Chan Fook Mun and Khin Mi Mi Aung. 2018. High-performance FV somewhat homomorphic encryption on GPUs: An implementation using CUDA. IACR Transactions on Cryptographic Hardware and Embedded Systems 2018 2 (2018) 70\u201395.","DOI":"10.46586\/tches.v2018.i2.70-95"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","unstructured":"Martin Albrecht Melissa Chase Hao Chen Jintai Ding Shafi Goldwasser Sergey Gorbunov Shai Halevi Jeffrey Hoffstein Kim Laine and Kristin Lauter. 2021. Homomorphic encryption standard. In Protecting Privacy through Homomorphic Encryption. Springer International Publishing 31\u201362. DOI:10.1007\/978-3-030-77287-1_2","DOI":"10.1007\/978-3-030-77287-1_2"},{"key":"e_1_3_2_8_2","doi-asserted-by":"crossref","first-page":"520","DOI":"10.1007\/978-3-662-64331-0_27","volume-title":"International Conference on Financial Cryptography and Data Security","author":"Alves Pedro Geraldo MR","year":"2021","unstructured":"Pedro Geraldo MR Alves, Jheyne N Ortiz, and Diego F Aranha. 2021. Faster homomorphic encryption over gpgpus via hierarchical dgt. In International Conference on Financial Cryptography and Data Security. Springer, 520\u2013540."},{"key":"e_1_3_2_9_2","first-page":"311","volume-title":"Conference on the Theory and Application of Cryptographic Techniques","author":"Barrett Paul","year":"1986","unstructured":"Paul Barrett. 1986. Implementing the rivest shamir and adleman public key encryption algorithm on a standard digital signal processor. In Conference on the Theory and Application of Cryptographic Techniques. Springer, 311\u2013323."},{"key":"e_1_3_2_10_2","doi-asserted-by":"crossref","unstructured":"Dan Boneh Eu-Jin Goh and Kobbi Nissim. 2005. Evaluating 2-DNF formulas on ciphertexts. In Theory of Cryptography: Second Theory of Cryptography Conference TCC 2005 Cambridge MA USA February 10-12 2005. Proceedings 2 Springer 325\u2013341.","DOI":"10.1007\/978-3-540-30576-7_18"},{"key":"e_1_3_2_11_2","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1007\/978-3-319-96878-0_17","volume-title":"Advances in Cryptology\u2013CRYPTO 2018: 38th Annual International Cryptology Conference, Santa Barbara, CA, USA, August 19\u201323, 2018, Proceedings, Part III 38","author":"Bourse Florian","year":"2018","unstructured":"Florian Bourse, Michele Minelli, Matthias Minihold, and Pascal Paillier. 2018. Fast homomorphic evaluation of deep discretized neural networks. In Advances in Cryptology\u2013CRYPTO 2018: 38th Annual International Cryptology Conference, Santa Barbara, CA, USA, August 19\u201323, 2018, Proceedings, Part III 38. Springer, 483\u2013512."},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/2633600"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-70694-8_15"},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1007\/978-3-030-64834-3_8","volume-title":"Advances in Cryptology\u2013ASIACRYPT 2020: 26th International Conference on the Theory and Application of Cryptology and Information Security, Daejeon, South Korea, December 7\u201311, 2020, Proceedings, Part II 26","author":"Cheon Jung Hee","year":"2020","unstructured":"Jung Hee Cheon, Dongwoo Kim, and Duhyeong Kim. 2020. Efficient homomorphic comparison methods with optimal complexity. In Advances in Cryptology\u2013ASIACRYPT 2020: 26th International Conference on the Theory and Application of Cryptology and Information Security, Daejeon, South Korea, December 7\u201311, 2020, Proceedings, Part II 26. Springer, 221\u2013256."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00145-019-09319-x"},{"key":"e_1_3_2_16_2","unstructured":"Edward Chou Josh Beal Daniel Levy Serena Yeung Albert Haque and Li Fei-Fei. 2018. Faster cryptonets: Leveraging sparsity for real-world encrypted inference. arXiv:1811.09953. Retrieved from https:\/\/arxiv.org\/abs\/1811.09953."},{"key":"e_1_3_2_17_2","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1007\/978-3-319-29172-7_11","volume-title":"Cryptography and Information Security in the Balkans: Second International Conference, BalkanCryptSec 2015, Koper, Slovenia, September 3-4, 2015, Revised Selected Papers 2","author":"Dai Wei","year":"2016","unstructured":"Wei Dai and Berk Sunar. 2016. cuHE: A homomorphic encryption accelerator library. In Cryptography and Information Security in the Balkans: Second International Conference, BalkanCryptSec 2015, Koper, Slovenia, September 3-4, 2015, Revised Selected Papers 2. Springer, 169\u2013186."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3314221.3314628"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2012.2211477"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-46800-5_24"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1985.1057074"},{"key":"e_1_3_2_22_2","unstructured":"Junfeng Fan and Frederik Vercauteren. 2012. Somewhat practical fully homomorphic encryption. IACR Cryptology ePrint Archive 2012 (2012) 144."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/1457838.1457864"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/1536414.1536440"},{"key":"e_1_3_2_25_2","first-page":"201","volume-title":"International Conference on Machine Learning","author":"Gilad-Bachrach Ran","year":"2016","unstructured":"Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. 2016. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. In International Conference on Machine Learning. PMLR, 201\u2013210."},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-12612-4_5"},{"issue":"1","key":"e_1_3_2_27_2","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1007\/s00145-020-09368-7","article-title":"Bootstrapping for helib","volume":"34","author":"Halevi Shai","year":"2021","unstructured":"Shai Halevi and Victor Shoup. 2021. Bootstrapping for helib. Journal of Cryptology 34, 1 (2021), 7.","journal-title":"Journal of Cryptology"},{"key":"e_1_3_2_28_2","volume-title":"Annual Computer Security Applications Conference (ACSAC 2016), Los Angeles, California, USA","volume":"11","author":"Hesamifard Ehsan","year":"2016","unstructured":"Ehsan Hesamifard, Hassan Takabi, and Mehdi Ghasemi. 2016. Cryptodl: Towards deep learning over encrypted data. In Annual Computer Security Applications Conference (ACSAC 2016), Los Angeles, California, USA, Vol. 11."},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3658644.3690375"},{"key":"e_1_3_2_30_2","doi-asserted-by":"crossref","unstructured":"Wonkyung Jung Sangpyo Kim Jung Ho Ahn Jung Hee Cheon and Younho Lee. 2021. Over 100x faster bootstrapping in fully homomorphic encryption through memory-centric optimization with GPUs. IACR Transactions on Cryptographic Hardware and Embedded Systems 2021 4 (2021) 114\u2013148.","DOI":"10.46586\/tches.v2021.i4.114-148"},{"key":"e_1_3_2_31_2","first-page":"1651","volume-title":"27th USENIX Security Symposium (USENIX Security 18)","author":"Juvekar Chiraag","year":"2018","unstructured":"Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan. 2018. GAZELLE: A low latency framework for secure neural network inference. In 27th USENIX Security Symposium (USENIX Security 18). 1651\u20131669."},{"key":"e_1_3_2_32_2","doi-asserted-by":"crossref","unstructured":"Donghwan Kim Jaiyoung Park Jongmin Kim Sangpyo Kim and Jung Ho Ahn. 2023. HyPHEN: A hybrid packing method and its optimizations for homomorphic encryption-based neural networks. IEEE Access 12 (2023) 3024\u20133038.","DOI":"10.1109\/ACCESS.2023.3348170"},{"key":"e_1_3_2_33_2","article-title":"Cheddar: A swift fully homomorphic encryption library for CUDA GPUs","author":"Kim Jongmin","year":"2024","unstructured":"Jongmin Kim, Wonseok Choi, and Jung Ho Ahn. 2024. Cheddar: A swift fully homomorphic encryption library for CUDA GPUs. CoRR (2024).","journal-title":"CoRR"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3579371.3589053"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO56248.2022.00086"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC50251.2020.00033"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3470496.3527415"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3133956.3134056"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/2535925"},{"key":"e_1_3_2_40_2","unstructured":"NVIDIA Corporation. 2024. NVIDIA Nsight Compute | NVIDIA Developer. Retrieved from https:\/\/developer.nvidia.com\/nsight-compute[Accessed on: 2024\/05\/23]."},{"key":"e_1_3_2_41_2","unstructured":"NVIDIA Corporation. 2024. NVIDIA Visual Profiler | NVIDIA Developer. Retrieved from https:\/\/developer.nvidia.com\/nvidia-visual-profiler[Accessed on: 2024\/05\/23]."},{"key":"e_1_3_2_42_2","doi-asserted-by":"crossref","unstructured":"Ali \u015eah \u00d6zcan Can Ayduman Enes Recep T\u00fcrko\u011flu and Erkay Sava\u015f. 2023. Homomorphic encryption on GPU. IEEE Access 11 (2023) 84168\u201384186.","DOI":"10.1109\/ACCESS.2023.3265583"},{"key":"e_1_3_2_43_2","unstructured":"Ali \u015eah \u00d6zcan and Erkay Sava\u015f. 2023. Two algorithms for fast GPU implementation of NTT. IACR Cryptology ePrint Archive 2023 (2023) 1410."},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11227-021-03980-5"},{"key":"e_1_3_2_45_2","article-title":"AESPA: Accuracy preserving low-degree polynomial activation for fast private inference","author":"Park Jaiyoung","year":"2022","unstructured":"Jaiyoung Park, Michael Jaemin Kim, Wonkyung Jung, and Jung Ho Ahn. 2022. AESPA: Accuracy preserving low-degree polynomial activation for fast private inference. CoRR (2022).","journal-title":"CoRR"},{"key":"e_1_3_2_46_2","doi-asserted-by":"crossref","DOI":"10.1142\/3254","volume-title":"Chinese Remainder Theorem: Applications in Computing, Coding, Cryptography","author":"Pei Dingyi","year":"1996","unstructured":"Dingyi Pei, Arto Salomaa, and Cunsheng Ding. 1996. Chinese Remainder Theorem: Applications in Computing, Coding, Cryptography. World Scientific."},{"issue":"11","key":"e_1_3_2_47_2","first-page":"169","article-title":"On data banks and privacy homomorphisms","volume":"4","author":"Rivest Ronald L.","year":"1978","unstructured":"Ronald L. Rivest, Len Adleman, and Michael L. Dertouzos. 1978. On data banks and privacy homomorphisms. Foundations of Secure Computation 4, 11 (1978), 169\u2013180.","journal-title":"Foundations of Secure Computation"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/359340.359342"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3470496.3527393"},{"key":"e_1_3_2_50_2","unstructured":"2023. Microsoft SEAL (release 4.1). Retrieved from https:\/\/github.com\/Microsoft\/SEAL"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613424.3614279"},{"key":"e_1_3_2_52_2","doi-asserted-by":"crossref","unstructured":"Nigel P. Smart and Frederik Vercauteren. 2014. Fully homomorphic SIMD operations. Designs Codes and Cryptography 71 (2014) 57\u201381.","DOI":"10.1007\/s10623-012-9720-4"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2022.3228628"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2024.3363900"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA56546.2023.10070984"}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3732942","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,2]],"date-time":"2025-07-02T12:20:47Z","timestamp":1751458847000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3732942"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,30]]},"references-count":54,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,6,30]]}},"alternative-id":["10.1145\/3732942"],"URL":"https:\/\/doi.org\/10.1145\/3732942","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"value":"1544-3566","type":"print"},{"value":"1544-3973","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,30]]},"assertion":[{"value":"2024-09-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-02","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}