{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,20]],"date-time":"2025-12-20T22:00:41Z","timestamp":1766268041975,"version":"3.41.0"},"reference-count":104,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2022,10,18]],"date-time":"2022-10-18T00:00:00Z","timestamp":1666051200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Embed. Comput. Syst."],"published-print":{"date-parts":[[2022,11,30]]},"abstract":"<jats:p>\n            Hardware acceleration of Artificial Intelligence (AI) workloads has gained widespread popularity with its potential to deliver unprecedented performance and efficiency. An important challenge remains in how AI accelerators are programmed to sustain high utilization without impacting end-user productivity. Prior software optimizations start with an input graph and focus on node-level optimizations,\n            <jats:italic>viz.<\/jats:italic>\n            dataflows and hierarchical tiling, and graph-level optimizations such as operation fusion. However, little effort has been devoted to inter-node on-chip scratchpad memory (SPM) management in Deep Learning (DL) accelerators, whose significance is bolstered by the recent trends in complex network topologies and the emergence of eager execution in DL frameworks.\n          <\/jats:p>\n          <jats:p>We characterize and show that there exists up to a 5.2\u00d7 performance gap in DL inference to be bridged using SPM management and propose OnSRAM, a novel SPM management framework integrated with the compiler runtime of a DL accelerator. We develop two variants, viz. \u00a0OnSRAM-Static, which works on static graphs to identify data structures that can be lucratively held on-chip based on their size, liveness and significance, and OnSRAM-Eager, which targets an eager execution model (no graph) and uses a history-based speculative scheme to hold\/discard data structures. We integrate OnSRAM \u00a0with TensorFlow and analyze it on multiple accelerator configurations. Across a suite of 12 images, objects, and language networks, on a 3\u00a0TFLOP system with a 2\u00a0MB SPM and 32\u00a0GBps external memory bandwidth, OnSRAM-Static \u00a0and OnSRAM-Eager \u00a0achieve 1.02\u20134.8\u00d7 and 1.02\u20133.1\u00d7 reduction in inference latency (batch size of 1), over a baseline with no SPM management. In terms of energy savings, we observe average reductions of 1.51\u00d7 (up to 4.1\u00d7) and 1.23\u00d7 (up to 2.9\u00d7) for the static and eager execution scenarios, respectively.<\/jats:p>","DOI":"10.1145\/3530909","type":"journal-article","created":{"date-parts":[[2022,4,27]],"date-time":"2022-04-27T11:53:27Z","timestamp":1651060407000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["OnSRAM: Efficient Inter-Node On-Chip Scratchpad Management in Deep Learning Accelerators"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1564-7443","authenticated-orcid":false,"given":"Subhankar","family":"Pal","sequence":"first","affiliation":[{"name":"IBM Thomas J. Watson Research Center, Yorktown Heights, NY"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0470-6364","authenticated-orcid":false,"given":"Swagath","family":"Venkataramani","sequence":"additional","affiliation":[{"name":"IBM Thomas J. Watson Research Center, Yorktown Heights, NY"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2593-8789","authenticated-orcid":false,"given":"Viji","family":"Srinivasan","sequence":"additional","affiliation":[{"name":"IBM Thomas J. Watson Research Center, Yorktown Heights, NY"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8952-0875","authenticated-orcid":false,"given":"Kailash","family":"Gopalakrishnan","sequence":"additional","affiliation":[{"name":"IBM Thomas J. Watson Research Center, Yorktown Heights, NY"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,10,18]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Google Research blog 2016 Google supercharges machine learning tasks with TPU custom chip"},{"key":"e_1_3_2_3_2","unstructured":"NVIDIA blog 2017 Nvidia tensor cores: Retrieved 1 January 2020 from https:\/\/www.nvidia.com\/en-us\/data-center\/tensorcore\/"},{"key":"e_1_3_2_4_2","first-page":"265","volume-title":"Proceedings of the OSDI","author":"Abadi Martin","year":"2016","unstructured":"Martin Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A system for large-scale machine learning. In Proceedings of the OSDI. 265\u2013283."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSCC42613.2021.9365791"},{"key":"e_1_3_2_6_2","unstructured":"Byung Hoon Ahn Jinwon Lee Jamie Menjay Lin Hsin-Pai Cheng Jilei Hou and Hadi Esmaeilzadeh. 2020. Ordering chaos: Memory-aware scheduling of irregularly wired neural networks for edge devices. In Proceedings of Machine Learning and Systems 2 (2020) 44\u201357."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2016.11"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO.2016.7783725"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSI.2017.2757036"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2019.2941875"},{"key":"e_1_3_2_11_2","volume-title":"Proceedings of the VLSI Symposium","author":"Gopalakrishnan B. Fleischer, S. Shukla, M. Ziegler, J. Silberman, J. Oh, V. Srinivasan, J. Choi, S. Mueller, A. Agrawal, T. Babinsky, N. Cao, C. Y. Chen, P. Chuang, T. Fox, G. Gristede, M. Guillorn, H. Haynie, M. Klaiber, D. Lee, S.Lo, G. Maier, M. Scheuermann, S. Venkataramani, C. Vezyrtzis, N. Wang, F. Yee, C. Zhou, P. F. Lu, B. Curran, L. Chang, and K.","year":"2018","unstructured":"B. Fleischer, S. Shukla, M. Ziegler, J. Silberman, J. Oh, V. Srinivasan, J. Choi, S. Mueller, A. Agrawal, T. Babinsky, N. Cao, C. Y. Chen, P. Chuang, T. Fox, G. Gristede, M. Guillorn, H. Haynie, M. Klaiber, D. Lee, S.Lo, G. Maier, M. Scheuermann, S. Venkataramani, C. Vezyrtzis, N. Wang, F. Yee, C. Zhou, P. F. Lu, B. Curran, L. Chang, and K. Gopalakrishnan. 2018. A scalable multi-TeraOPS deep learning processor core for AI training and inference. In Proceedings of the VLSI Symposium."},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3320060"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/177492.177575"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/300979.300985"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/1816038.1815993"},{"key":"e_1_3_2_16_2","unstructured":"Chia-Yu Chen Jungwook Choi Daniel Brand Ankur Agrawal Wei Zhang and Kailash Gopalakrishnan. 2018. AdaComp: adaptive residual gradient compression for data-parallel distributed training. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence (AAAI\u201918\/IAAI\u201918\/EAAI\u201918) . AAAI Press Article 345 2827\u20132835."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/2541940.2541967"},{"key":"e_1_3_2_18_2","first-page":"578","volume-title":"Proceedings of the OSDI","author":"Chen Tianqi","year":"2018","unstructured":"Tianqi Chen, Thierry Moreau, Ziheng Jiang, Haichen Shen, Eddie Q. Yan, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. TVM: An automated end-to-end optimizing compiler for deep learning. In Proceedings of the OSDI. 578\u2013594."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/191995.192030"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2016.40"},{"key":"e_1_3_2_21_2","unstructured":"Jungwook Choi Zhuo Wang Swagath Venkataramani Pierce I-Jen Chuang Vijayalakshmi Srinivasan and Kailash Gopalakrishnan. 2018. PACT: Parameterized clipping activation for quantized neural networks. arXiv:1805.06085. Retrieved from https:\/\/arxiv.org\/abs\/1805.06085."},{"key":"e_1_3_2_22_2","unstructured":"Matthieu Courbariaux Yoshua Bengio and Jean-Pierre David. 2015. Binaryconnect: Training deep neural networks with binary weights during propagations. Advances in Neural Information Processing Systems 28 (2015)."},{"key":"e_1_3_2_23_2","unstructured":"Scott Cyphers Arjun K. Bansal Anahita Bhiwandiwalla Jayaram Bobba Matthew Brookhart Avijit Chakraborty William Constable Christian Convey Leona Cook Omar Kanawi Robert Kimball Jason Knight Nikolay Korovaiko Varun Kumar Yixing Lao Christopher R. Lishka Jaikrishnan Menon Jennifer Myers Sandeep Aswath Narayana Adam Procter and Tristan J. Webb. 2018. Intel nGraph: An intermediate representation compiler and executor for deep learning. arXiv:1801.08058. Retrieved from https:\/\/arxiv.org\/abs\/1801.08058."},{"key":"e_1_3_2_24_2","unstructured":"Scott Cyphers Arjun K. Bansal Anahita Bhiwandiwalla Jayaram Bobba Matthew Brookhart Avijit Chakraborty Will Constable Christian Convey Leona Cook Omar Kanawi Robert Kimball Jason Knight Nikolay Korovaiko Varun Kumar Yixing Lao Christopher R. Lishka Jaikrishnan Menon Jennifer Myers Sandeep Aswath Narayana Adam Procter and Tristan J. Webb. 2018. Intel ngraph: An intermediate representation compiler and executor for deep learning. arXiv:1801.08058. Retrieved from https:\/\/arxiv.org\/abs\/1801.08058."},{"key":"e_1_3_2_25_2","unstructured":"Dipankar Das Sasikanth Avancha Dheevatsa Mudigere Karthikeyan Vaidynathan Srinivas Sridharan Dhiraj Kalamkar Bharat Kaul and Pradeep Dubey. 2016. Distributed deep learning using synchronous stochastic gradient descent. arXiv preprint arXiv:1602.06709 (2016)."},{"key":"e_1_3_2_26_2","first-page":"1232","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 25","author":"Dean Jeffrey","year":"2012","unstructured":"Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Marc\u2019aurelio Ranzato, Andrew Senior, Paul Tucker, Ke Yang, Quoc V. Le, and Andrew Y. Ng. 2012. Large scale distributed deep networks. In Proceedings of the Advances in Neural Information Processing Systems 25. P. Bartlett, F.c.n. Pereira, C.j.c. Burges, L. Bottou, and K.q. Weinberger (Eds.), 1232\u20131240. Retrieved from http:\/\/books.nips.cc\/papers\/files\/nips25\/NIPS2012_0598.pdf."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/PACT.2015.21"},{"volume-title":"Proceedings of the CVPR","year":"2015","key":"e_1_3_2_29_2","unstructured":"Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the CVPR."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2011.5981829"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2018.00012"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/JETCAS.2020.3040300"},{"key":"e_1_3_2_33_2","series-title":"Proceedings of the 35th International Conference on Machine Learning","first-page":"1662","volume":"80","author":"Gao Yuanxiang","year":"2018","unstructured":"Yuanxiang Gao, Li Chen, and Baochun Li. 2018. Spotlight: Optimizing device placement for training deep neural networks. In Proceedings of the 35th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 80). Jennifer Dy and Andreas Krause (Eds.), PMLR, Stockholmsmassan, Stockholm Sweden, 1662\u20131670. Retrieved from http:\/\/proceedings.mlr.press\/v80\/gao18a.html."},{"volume-title":"Google AI Blog: Eager Execution: An imperative, define-by-run interface to TensorFlow","year":"2017","key":"e_1_3_2_34_2","unstructured":"Google. 2017. Google AI Blog: Eager Execution: An imperative, define-by-run interface to TensorFlow. Retrieved 1 January 2020 from http:\/\/ai.googleblog.com\/2017\/10\/eager-execution-imperative-define-by.html."},{"key":"e_1_3_2_35_2","unstructured":"Suyog Gupta Ankur Agrawal Kailash Gopalakrishnan and Pritish Narayanan. 2015. Deep learning with limited numerical precision. In International Conference on Machine Learning PMLR 1737\u20131746."},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC47752.2019.9041955"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2016.30"},{"key":"e_1_3_2_38_2","unstructured":"Song Han Huizi Mao and William J. Dally. 2015. Deep compression: Compressing deep neural network with pruning trained quantization and huffman coding. arXiv:1510.00149. Retrieved from https:\/\/arxiv.org\/abs\/1510.00149."},{"key":"e_1_3_2_39_2","unstructured":"Kaiming He Xiangyu Zhang Shaoqing Ren and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 770\u2013778."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3392717.3392751"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3282307"},{"key":"e_1_3_2_42_2","unstructured":"Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand Marco Andreetto and Hartwig Adam. 2017. MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861. Retrieved from https:\/\/arxiv.org\/abs\/1704.04861."},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3352460.3358263"},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","unstructured":"Gao Huang Zhuang Liu Laurens Van Der Maaten and Kilian Q. Weinberger. 2017. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 4700\u20134708.","DOI":"10.1109\/CVPR.2017.243"},{"key":"e_1_3_2_45_2","doi-asserted-by":"crossref","unstructured":"Forrest Iandola N. Matthew W. Moskewicz Khalid Ashraf and Kurt Keutzer. 2016. Firecaffe: near-linear acceleration of deep neural network training on compute clusters. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2592\u20132600.","DOI":"10.1109\/CVPR.2016.284"},{"key":"e_1_3_2_46_2","unstructured":"Forrest N. Iandola Song Han Matthew W. Moskewicz Khalid Ashraf William J. Dally and Kurt Keutzer. 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size. arXiv:1602.07360. Retrieved from https:\/\/arxiv.org\/abs\/1602.07360."},{"key":"e_1_3_2_47_2","doi-asserted-by":"crossref","unstructured":"Max Jaderberg Andrea Vedaldi and Andrew Zisserman. 2014. Speeding up convolutional neural networks with low rank expansions. arXiv:1405.3866. Retrieved from https:\/\/arxiv.org\/abs\/1405.3866.","DOI":"10.5244\/C.28.88"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3195970.3196012"},{"key":"e_1_3_2_49_2","first-page":"463","volume-title":"Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation.","author":"Jiang Yimin","year":"2020","unstructured":"Yimin Jiang, Yibo Zhu, Chang Lan, Bairen Yi, Yong Cui, and Chuanxiong Guo. 2020. A unified architecture for accelerating distributed \\( \\lbrace \\) DNN \\( \\rbrace \\) training in heterogeneous GPU\/CPU clusters. In Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation.463\u2013479."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2016.41"},{"key":"e_1_3_2_51_2","unstructured":"Alex Krizhevsky. 2014. One weird trick for parallelizing convolutional neural networks. arXiv:1404.5997. Retrieved from https:\/\/arxiv.org\/abs\/1404.5997."},{"key":"e_1_3_2_52_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proceedings of the Advances in Neural Information Processing Systems."},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3352460.3358252"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3173162.3173176"},{"key":"e_1_3_2_55_2","doi-asserted-by":"crossref","unstructured":"Jiansong Li Wei Cao Xiao Dong Guangli Li Xueying Wang Peng Zhao Lei Liu and Xiaobing Feng. 2021. Compiler-assisted Operator Template Library for DNN Accelerators. International Journal of Parallel Programming 49 5 (2021) 628\u2013645.","DOI":"10.1007\/s10766-021-00701-6"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2016.42"},{"key":"e_1_3_2_58_2","doi-asserted-by":"crossref","unstructured":"Wei Liu Dragomir Anguelov Dumitru Erhan Christian Szegedy Scott Reed Cheng-Yang Fu and Alexander C. Berg. 2016. Ssd: Single shot multibox detector. In European Conference on Computer Vision Springer Cham 21\u201337.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"e_1_3_2_59_2","unstructured":"Sangkug Lym Armand Behroozi Wei Wen Ge Li Yongkee Kwon and Mattan Erez. 2019. Mini-batch serialization: Cnn training with inter-layer data reuse. Proceedings of Machine Learning and Systems 1 (2019) 264\u2013275."},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/2133382.2133388"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO.1999.809461"},{"key":"e_1_3_2_62_2","unstructured":"Azalia Mirhoseini Anna Goldie Hieu Pham Benoit Steiner Quoc V. Le and Jeff Dean. 2018. Hierarchical planning for device placement. Retrieved from https:\/\/openreview.net\/pdf?id=Hkc-TeZ0W."},{"key":"e_1_3_2_63_2","unstructured":"Azalia Mirhoseini Hieu Pham Quoc V. Le Benoit Steiner Rasmus Larsen Yuefeng Zhou Naveen Kumar Mohammad Norouzi Samy Bengio and Jeff Dean. 2017. Device placement optimization with reinforcement learning. In International Conference on Machine Learning PMLR 2430\u20132439."},{"key":"e_1_3_2_64_2","volume-title":"Proceedings of the 44th International Symposium on Computer Architecture.","author":"Yoon Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, Rick Boyle, Pierre-luc Cantin, Clifford Chao, Chris Clark, Jeremy Coriell, Mike Daley, Matt Dau, Jeffrey Dean, Ben Gelb, Tara Vazir Ghaemmaghami, Rajendra Gottipati, William Gulland, Robert Hagmann, C. Richard Ho, Doug Hogberg, John Hu, Robert Hundt, Dan Hurt, Julian Ibarz, Aaron Jaffey, Alek Jaworski, Alexander Kaplan, Harshit Khaitan, Andy Koch, Naveen Kumar, Steve Lacy, James Laudon, James Law, Diemthu Le, Chris Leary, Zhuyuan Liu, Kyle Lucke, Alan Lundin, Gordon MacKean, Adriana Maggiore, Maire Mahony, Kieran Miller, Rahul Nagarajan, Ravi Narayanaswami, Ray Ni, Kathy Nix, Thomas Norrie, Mark Omernick, Narayana Penukonda, Andy Phelps, Jonathan Ross, Matt Ross, Amir Salek, Emad Samadiani, Chris Severn, Gregory Sizikov, Matthew Snelham, Jed Souter, Dan Steinberg, Andy Swing, Mercedes Tan, Gregory Thorson, Bo Tian, Horia Toma, Erick Tuttle, Vijay Vasudevan, Richard Walter, Walter Wang, Eric Wilcox, and Doe Hyun","year":"2017","unstructured":"Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, Rick Boyle, Pierre-luc Cantin, Clifford Chao, Chris Clark, Jeremy Coriell, Mike Daley, Matt Dau, Jeffrey Dean, Ben Gelb, Tara Vazir Ghaemmaghami, Rajendra Gottipati, William Gulland, Robert Hagmann, C. Richard Ho, Doug Hogberg, John Hu, Robert Hundt, Dan Hurt, Julian Ibarz, Aaron Jaffey, Alek Jaworski, Alexander Kaplan, Harshit Khaitan, Andy Koch, Naveen Kumar, Steve Lacy, James Laudon, James Law, Diemthu Le, Chris Leary, Zhuyuan Liu, Kyle Lucke, Alan Lundin, Gordon MacKean, Adriana Maggiore, Maire Mahony, Kieran Miller, Rahul Nagarajan, Ravi Narayanaswami, Ray Ni, Kathy Nix, Thomas Norrie, Mark Omernick, Narayana Penukonda, Andy Phelps, Jonathan Ross, Matt Ross, Amir Salek, Emad Samadiani, Chris Severn, Gregory Sizikov, Matthew Snelham, Jed Souter, Dan Steinberg, Andy Swing, Mercedes Tan, Gregory Thorson, Bo Tian, Horia Toma, Erick Tuttle, Vijay Vasudevan, Richard Walter, Walter Wang, Eric Wilcox, and Doe Hyun Yoon. 2017. In-datacenter performance analysis of a tensor processing unit. In Proceedings of the 44th International Symposium on Computer Architecture."},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3123939.3124545"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3466752.3480134"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2018.00067"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/3410463.3414627"},{"key":"e_1_3_2_69_2","first-page":"C150\u2013C151","volume-title":"Proceedings of the 2019 Symposium on VLSI Technology","year":"2019","unstructured":"Subhankar Pal, Dong-hyeon Park, Siying Feng, Paul Gao, Jielun Tan, Austin Rovinski, Shaolin Xie, Chun Zhao, Aporva Amarnath, Timothy Wesley, Jonathan Beaumont, Kuan-Yu Chen, Chaitali Chakrabarti, Michael Taylor, Trevor Mudge, David Blaauw, Hun-Seok Kim, and Ronald Dreslinski. 2019. A 7.3 m output non-zeros\/j sparse matrix-matrix multiplication accelerator using memory reconfiguration in 40 nm. In Proceedings of the 2019 Symposium on VLSI Technology. IEEE, C150\u2013C151."},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS51385.2021.00046"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS.2019.00042"},{"key":"e_1_3_2_72_2","doi-asserted-by":"crossref","unstructured":"Angshuman Parashar Minsoo Rhu Anurag Mukkara Antonio Puglielli Rangharajan Venkatesan Brucek Khailany Joel Emer Stephen W. Keckler and William J. Dally. 2017. SCNN: An accelerator for compressed-sparse convolutional neural networks. ACM SIGARCH Computer Architecture News 45 2 (2017) 27\u201340.","DOI":"10.1145\/3140659.3080254"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSSC.2019.2960480"},{"key":"e_1_3_2_74_2","first-page":"8026","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas K\u00f6pf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. Pytorch: An imperative style, high-performance deep learning library. In Proceedings of the Advances in Neural Information Processing Systems. 8026\u20138037."},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3297858.3304025"},{"key":"e_1_3_2_76_2","doi-asserted-by":"crossref","unstructured":"Mohammad Rastegari Vicente Ordonez Joseph Redmon and Ali Farhadi. 2016. Xnor-net: Imagenet classification using binary convolutional neural networks. In European Conference on Computer Vision Springer Cham 525\u2013542.","DOI":"10.1007\/978-3-319-46493-0_32"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2016.32"},{"key":"e_1_3_2_78_2","doi-asserted-by":"crossref","unstructured":"Minsoo Rhu Natalia Gimelshein Jason Clemons Arslan Zulfiqar and Stephen W. Keckler. 2016. VDNN: virtualized deep neural networks for scalable memory-efficient neural network design. In The 49th Annual IEEE\/ACM International Symposium on Microarchitecture (MICRO-49) . IEEE Press Article 18 1\u201313.","DOI":"10.1109\/MICRO.2016.7783721"},{"key":"e_1_3_2_79_2","unstructured":"Nadav Rotem Jordan Fix Saleem Abdulrasool Summer Deng Roman Dzhabarov James Hegeman Roman Levenstein Bert Maher Nadathur Satish Jakob Olesen Jongsoo Park Artem Rakhov and Misha Smelyanskiy. 2018. Glow: Graph lowering compiler techniques for neural networks. arXiv:1805.00907. Retrieved from https:\/\/arxiv.org\/abs\/1805.00907."},{"key":"e_1_3_2_80_2","doi-asserted-by":"crossref","unstructured":"Frank Seide Hao Fu Jasha Droppo Gang Li and Dong Yu. 2014. 1-Bit stochastic gradient descent and application to data-parallel distributed training of speech DNNs. In Proceedings of the 15th Annual Conference of the International Speech Communication Association.","DOI":"10.21437\/Interspeech.2014-274"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2018.2879434"},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/3299874.3317979"},{"key":"e_1_3_2_83_2","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556. Retrieved from https:\/\/arxiv.org\/abs\/1409.1556."},{"key":"e_1_3_2_84_2","volume-title":"Proceedings of the NeurIPS","author":"Steiner Benoit","year":"2019","unstructured":"Benoit Steiner, Zachary Devito, Soumith Chintala, Sam Gross, Adam Paszke, Francisco Massa, Adam Lerer, Gregory Chanan, Zeming Lin, Edward Yang, Alban Desmaison, Alykhan Tejani, Andreas Dipl.-Ing. Kopf, James Bradbury, Luca Antiga, Martin Raison, Natalia Gimelshein, Sasank Chilamkurthy, Trevor Killeen, Lu Fang, and Junjie Bai. 2019. PyTorch: An imperative style, high-performance deep learning library. In Proceedings of the NeurIPS."},{"key":"e_1_3_2_85_2","first-page":"3104","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Sutskever Ilya","year":"2014","unstructured":"Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the Advances in Neural Information Processing Systems. 3104\u20133112."},{"key":"e_1_3_2_86_2","doi-asserted-by":"crossref","unstructured":"Christian Szegedy Sergey Ioffe Vincent Vanhoucke and Alexander A. Alemi. 2017. Inception-v4 inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI Conference on Artificial Intelligence .","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"e_1_3_2_87_2","doi-asserted-by":"crossref","unstructured":"Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jon Shlens and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2818\u20132826.","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_3_2_88_2","doi-asserted-by":"crossref","unstructured":"Xiaohan Tao Jianmin Pang Jinlong Xu and Yu Zhu. 2021. Compiler-directed scratchpad memory data transfer optimization for multithreaded applications on a heterogeneous many-core architecture. The Journal of Supercomputing 77 12 (2021) 14502\u201314524.","DOI":"10.1007\/s11227-021-03853-x"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-010-0201-1_1"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1109\/12.381947"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295349"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1145\/2540708.2540710"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1109\/PACT.2017.39"},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.1109\/MM.2019.2931584"},{"key":"e_1_3_2_95_2","doi-asserted-by":"publisher","DOI":"10.1145\/3079856.3080244"},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","DOI":"10.1145\/2627369.2627613"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1109\/HiPC.2019.00036"},{"key":"e_1_3_2_98_2","first-page":"153","volume-title":"Proceedings of the 2021 ACM\/IEEE 48th Annual International Symposium on Computer Architecture","year":"2021","unstructured":"Swagath Venkataramani, Vijayalakshmi Srinivasan, Wei Wang, Sanchari Sen, Jintao Zhang, Ankur Agrawal, Monodeep Kar, Shubham Jain, Alberto Mannari, Hoang Tran, Yulong Li, Eri Ogawa, Kazuaki Ishizaki, Hiroshi Inoue, Marcel Schaal, Mauricio Serrano, Jungwook Choi, Xiao Sun, Naigang Wang, Chia-Yu Chen, Allison Allain, James Bonano, Nianzheng Cao, Robert Casatuta, Matthew Cohen, Bruce Fleischer, Michael Guillorn, Howard Haynie, Jinwook Jung, Mingu Kang, Kyu-hyoun Kim, Siyu Koswatta, Saekyu Lee, Martin Lutz, Silvia Mueller, Jinwook Oh, Ashish Ranjan, Zhibin Ren, Scot Rider, Kerstin Schelm, Michael Scheuermann, Joel Silberman, Jie Yang, Vidhi Zalani, Xin Zhang, Ching Zhou, Matt Ziegler, Vinay Shah, Moriyoshi Ohara, Pong-Fei Lu, Brian Curran, Sunil Shukla, Leland Chang, and Kailash Gopalakrishnan. 2021. RaPiD: AI accelerator for ultra-low precision training and inference. In Proceedings of the 2021 ACM\/IEEE 48th Annual International Symposium on Computer Architecture. IEEE, 153\u2013166."},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.5555\/266800.266827"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.5555\/266800.266827"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.1109\/HOTCHIPS.2019.8875671"},{"key":"e_1_3_2_102_2","unstructured":"Saining Xie Ross Girshick Piotr Doll\u00e1r Zhuowen Tu and Kaiming He. 2017. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 1492\u20131500."},{"key":"e_1_3_2_103_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCAS45731.2020.9180871"},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS49936.2021.00024"},{"key":"e_1_3_2_105_2","unstructured":"Barret Zoph and Quoc V. Le. 2017. Neural architecture search with reinforcement learning. arXiv:1611.01578. Retrieved from https:\/\/arxiv.org\/abs\/1611.01578"}],"container-title":["ACM Transactions on Embedded Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3530909","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3530909","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:09:27Z","timestamp":1750183767000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3530909"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,18]]},"references-count":104,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,11,30]]}},"alternative-id":["10.1145\/3530909"],"URL":"https:\/\/doi.org\/10.1145\/3530909","relation":{},"ISSN":["1539-9087","1558-3465"],"issn-type":[{"type":"print","value":"1539-9087"},{"type":"electronic","value":"1558-3465"}],"subject":[],"published":{"date-parts":[[2022,10,18]]},"assertion":[{"value":"2021-07-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-04-08","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-10-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}