{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T14:57:56Z","timestamp":1787237876716,"version":"build-2736575974"},"reference-count":90,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2022,12,22]],"date-time":"2022-12-22T00:00:00Z","timestamp":1671667200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"United Kingdom EPSRC","award":["EP\/V028251\/1, EP\/L016796\/1, EP\/N031768\/1, EP\/P010040\/1, EP\/S030069\/1"],"award-info":[{"award-number":["EP\/V028251\/1, EP\/L016796\/1, EP\/N031768\/1, EP\/P010040\/1, EP\/S030069\/1"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Reconfigurable Technol. Syst."],"published-print":{"date-parts":[[2023,3,31]]},"abstract":"<jats:p>This work introduces Remarn, a reconfigurable multi-threaded multi-core accelerator supporting both spatial and temporal co-execution of Recurrent Neural Network (RNN) inferences. It increases processing capabilities and quality of service of cloud-based neural processing units (NPUs) by improving their hardware utilization and by reducing design latency, with two innovations. First, a custom coarse-grained multi-threaded RNN\/Long Short-Term Memory (LSTM) hardware architecture, switching tasks among threads when RNN computational engines meet data hazards. Second, the partitioning of this hardware architecture into multiple full-fledged sub-accelerator cores, enabling spatially co-execution of multiple RNN\/LSTM inferences. These innovations improve the exploitation of the available parallelism to increase runtime hardware utilization and boost design throughput. Evaluation results show that a dual-threaded quad-core Remarn NPU achieves 2.91 times higher performance while only occupying 5.0% more area than a single-threaded one on a Stratix 10 FPGA. When compared with a Tesla V100 GPU implementation, our design achieves 6.5 times better performance and 15.6 times higher power efficiency, showing that our approach contributes to high performance and energy-efficient FPGA-based multi-RNN inference designs for datacenters.<\/jats:p>","DOI":"10.1145\/3534969","type":"journal-article","created":{"date-parts":[[2022,5,17]],"date-time":"2022-05-17T09:31:12Z","timestamp":1652779872000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Remarn: A Reconfigurable Multi-threaded Multi-core Accelerator for Recurrent Neural Networks"],"prefix":"10.1145","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9263-6529","authenticated-orcid":false,"given":"Zhiqiang","family":"Que","sequence":"first","affiliation":[{"name":"Imperial College London, South Kensington, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5701-7466","authenticated-orcid":false,"given":"Hiroki","family":"Nakahara","sequence":"additional","affiliation":[{"name":"Tokyo Institute of Technology, Ohokayama, Tokyo, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2387-5611","authenticated-orcid":false,"given":"Hongxiang","family":"Fan","sequence":"additional","affiliation":[{"name":"Imperial College London, South Kensington, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1540-189X","authenticated-orcid":false,"given":"He","family":"Li","sequence":"additional","affiliation":[{"name":"University of Cambridge, Cambridge, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9261-2300","authenticated-orcid":false,"given":"Jiuxi","family":"Meng","sequence":"additional","affiliation":[{"name":"Imperial College London, South Kensington, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6856-6727","authenticated-orcid":false,"given":"Kuen Hung","family":"Tsoi","sequence":"additional","affiliation":[{"name":"Corerain Technologies Ltd., Shenzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0202-9408","authenticated-orcid":false,"given":"Xinyu","family":"Niu","sequence":"additional","affiliation":[{"name":"Corerain Technologies Ltd., Shenzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2347-9590","authenticated-orcid":false,"given":"Eriko","family":"Nurvitadhi","sequence":"additional","affiliation":[{"name":"Intel Corporation, Jones Farm Campus, Hillsboro, OR, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6750-927X","authenticated-orcid":false,"given":"Wayne","family":"Luk","sequence":"additional","affiliation":[{"name":"Imperial College London, South Kensington, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,12,22]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/FPL.2018.00077"},{"key":"e_1_3_1_3_2","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Amodei Dario","year":"2016","unstructured":"Dario Amodei et\u00a0al. 2016. Deep speech 2: End-to-end speech recognition in english and mandarin. In Proceedings of the International Conference on Machine Learning."},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.4236\/cs.2017.86010"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA45697.2020.00081"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVLSI.2019.2947639"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1147\/rd.446.0885"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/FPL.2018.00014"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICFPT51103.2020.00011"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3289602.3293898"},{"key":"e_1_3_1_11_2","article-title":"Recurrent neural networks hardware implementation on FPGA","author":"Chang Andre Xian Ming","year":"2015","unstructured":"Andre Xian Ming Chang, Berin Martini, and Eugenio Culurciello. 2015. Recurrent neural networks hardware implementation on FPGA. arXiv:1511.05552. Retrieved from https:\/\/arxiv.org\/abs\/1511.05552.","journal-title":"arXiv:1511.05552"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3370748.3406552"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/349500"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3218603.3218637"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA47549.2020.00027"},{"key":"e_1_3_1_16_2","unstructured":"Fran\u00e7ois Chollet et\u00a0al. 2015. Keras: Deep Learning Library for theano and tensorflow. https:\/\/keras.io\/k."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1049\/ip-cdt:20050177"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298878"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICFPT52863.2021.9609847"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2018.00012"},{"key":"e_1_3_1_21_2","article-title":"Spartus: A 9.4 TOp\/s FPGA-based LSTM accelerator exploiting spatio-temporal sparsity","author":"Gao Chang","year":"2021","unstructured":"Chang Gao, Tobi Delbruck, and Shih-Chii Liu. 2021. Spartus: A 9.4 TOp\/s FPGA-based LSTM accelerator exploiting spatio-temporal sparsity. IEEE Transactions on Neural Networks and Learning Systems. (Early Access)","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3174243.3174261"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3190508.3190541"},{"key":"e_1_3_1_24_2","article-title":"BRDS: An FPGA-based LSTM accelerator with row-balanced dual-ratio sparsification","author":"Ghasemzadeh Seyed Abolfazl","year":"2021","unstructured":"Seyed Abolfazl Ghasemzadeh, Erfan Bank Tavakoli, Mehdi Kamal, Ali Afzali-Kusha, and Massoud Pedram. 2021. BRDS: An FPGA-based LSTM accelerator with row-balanced dual-ratio sparsification. arXiv:2101.02667. Retrieved from https:\/\/arxiv.org\/abs\/2101.02667.","journal-title":"arXiv:2101.02667"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO50266.2020.00062"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.5555\/3176748.3176757"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/FCCM.2017.25"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASPDAC.2017.7858394"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3020078.3021745"},{"key":"e_1_3_1_30_2","article-title":"Deep speech: Scaling up end-to-end speech recognition","author":"Hannun Awni","year":"2014","unstructured":"Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, et\u00a0al. 2014. Deep speech: Scaling up end-to-end speech recognition. arXiv:1412.5567. Retrieved from https:\/\/arxiv.org\/abs\/1412.5567.","journal-title":"arXiv:1412.5567"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219845"},{"key":"e_1_3_1_33_2","volume-title":"White Paper 01231","unstructured":"Intel. [n.d.]. Understanding How Hyperflex Architecture Enables High Performance Systems. White Paper 01231."},{"key":"e_1_3_1_34_2","unstructured":"Intel. 2020. Intel Agilex Variable Precision DSP Blocks User Guide ."},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.micpro.2021.104417"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3079856.3080246"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA53966.2022.00065"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCAS51556.2021.9401395"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics10111249"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.5555\/3437539.3437732"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA51647.2021.00016"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/FPL.2019.00027"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2019.00028"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVLSI.2019.2920152"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3457886"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.5555\/2002472.2002491"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/MM.2005.34"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sysarc.2020.101839"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCAS45731.2020.9180869"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICFPT47387.2019.00054"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/1837274.1837356"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/FCCM.2019.00035"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/FPL.2016.7577314"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA51647.2021.00056"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3370748.3406561"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCSoC.2019.00042"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3079856.3080256"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/FCCM48280.2020.00011"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICFPT47387.2019.00072"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICFPT51103.2020.00012"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVLSI.2021.3135353"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASAP.2019.00-42"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASAP52443.2021.00025"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11265-020-01549-8"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICFPT51103.2020.00010"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-78890-6_1"},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/FPL.2018.00024"},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11265-020-01554-x"},{"key":"e_1_3_1_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3373087.3375301"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.5555\/3130379.3130707"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3316781.3317813"},{"key":"e_1_3_1_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/LCA.2019.2924007"},{"key":"e_1_3_1_73_2","doi-asserted-by":"crossref","unstructured":"Franyell Silfa Jose Maria Arnau and Antonio Gonzalez. 2020. E-BATCH: Energy-efficient and high-throughput RNN batching. ACM Transactions on Architecture and Code Optimization (TACO) 19 1 (2020) 1\u201323.","DOI":"10.1145\/3499757"},{"key":"e_1_3_1_74_2","first-page":"39","volume-title":"Advanced Computer Architecture","author":"Smith Burton J.","year":"1986","unstructured":"Burton J. Smith. 1986. A pipelined, shared resource MIMD computer. In Advanced Computer Architecture. 39\u201341."},{"key":"e_1_3_1_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3337801.3337804"},{"key":"e_1_3_1_76_2","doi-asserted-by":"publisher","DOI":"10.1109\/SmartCloud.2018.00009"},{"key":"e_1_3_1_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/FPT.2018.00038"},{"key":"e_1_3_1_78_2","first-page":"33","volume-title":"Proceedings of the Fall Joint Computer Conference, Part II: Very High Speed Computer Systems","author":"Thornton James E.","year":"1964","unstructured":"James E. Thornton. 1964. Parallel operation in the control data 6600. In Proceedings of the Fall Joint Computer Conference, Part II: Very High Speed Computer Systems. 33\u201340."},{"key":"e_1_3_1_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/FPL.2018.00072"},{"key":"e_1_3_1_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298935"},{"key":"e_1_3_1_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/3174243.3174253"},{"key":"e_1_3_1_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVLSI.2017.2717950"},{"key":"e_1_3_1_83_2","unstructured":"Zhao Wang Guangyu Sun Jingchen Zhu Zhe Zhou Yijiang Guo and Zhihang Yuan. 2021. METRO: A software-hardware co-design of interconnections for spatial DNN accelerators. arxiv:2108.10570 [cs.AR]. Retrieved from https:\/\/arxiv.org\/abs\/2108.10570."},{"key":"e_1_3_1_84_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVLSI.2019.2927375"},{"key":"e_1_3_1_85_2","unstructured":"Xilinx. 2017. Deep Learning with INT8 Optimization on Xilinx Devices. Retrieved from https:\/\/www.xilinx.com\/support\/documentation\/whitepapers\/wp486-deep-learning-int8.pdf."},{"key":"e_1_3_1_86_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVLSI.2019.2941921"},{"key":"e_1_3_1_87_2","article-title":"LSTM-sharp: An adaptable, energy-efficient hardware accelerator for long short-term memory","author":"Yazdani Reza","year":"2019","unstructured":"Reza Yazdani, Olatunji Ruwase, Minjia Zhang, Yuxiong He, Jose-Maria Arnau, and Antonio Gonz\u00e1lez. 2019. LSTM-sharp: An adaptable, energy-efficient hardware accelerator for long short-term memory. arXiv:1911.01258. Retrieved from https:\/\/arxiv.org\/abs\/1911.01258.","journal-title":"arXiv:1911.01258"},{"key":"e_1_3_1_88_2","first-page":"4694","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Ng Joe Yue-Hei","year":"2015","unstructured":"Joe Yue-Hei Ng, Matthew Hausknecht, Sudheendra Vijayanarasimhan, Oriol Vinyals, Rajat Monga, and George Toderici. 2015. Beyond short snippets: Deep networks for video classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4694\u20134702."},{"key":"e_1_3_1_89_2","article-title":"Recurrent neural network regularization","author":"Zaremba Wojciech","year":"2014","unstructured":"Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent neural network regularization. arXiv:1409.2329. Retrieved from https:\/\/arxiv.org\/abs\/1409.2329.","journal-title":"arXiv:1409.2329"},{"key":"e_1_3_1_90_2","first-page":"166","article-title":"Serving recurrent neural networks efficiently with a spatial accelerator","volume":"1","author":"Zhao Tian","year":"2019","unstructured":"Tian Zhao, Yaqi Zhang, and Kunle Olukotun. 2019. Serving recurrent neural networks efficiently with a spatial accelerator. Proc. Mach. Learn. Syst. 1 (2019), 166\u2013177.","journal-title":"Proc. Mach. Learn. Syst."},{"key":"e_1_3_1_91_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics10080882"}],"container-title":["ACM Transactions on Reconfigurable Technology and Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3534969","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3534969","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T15:02:54Z","timestamp":1750172574000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3534969"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,22]]},"references-count":90,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,3,31]]}},"alternative-id":["10.1145\/3534969"],"URL":"https:\/\/doi.org\/10.1145\/3534969","relation":{},"ISSN":["1936-7406","1936-7414"],"issn-type":[{"value":"1936-7406","type":"print"},{"value":"1936-7414","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,22]]},"assertion":[{"value":"2021-09-02","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-12-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}