{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T06:24:10Z","timestamp":1780467850773,"version":"3.54.1"},"reference-count":45,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,11,20]],"date-time":"2024-11-20T00:00:00Z","timestamp":1732060800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2024,12,31]]},"abstract":"<jats:p>Bit-serial Processing-In-Memory (PIM) is an attractive paradigm for accelerator architectures, for parallel workloads such as Deep Learning (DL), because of its capability to achieve massive data parallelism at a low area overhead and provide orders-of-magnitude data movement savings by moving computational resources closer to the data. While many PIM architectures have been proposed, improvements are needed in communicating intermediate results to consumer kernels, for communication between tiles at scale, for reduction operations, and for efficiently performing bit-serial operations with constants. We present PIMSAB, a scalable architecture that provides a spatially aware communication network for efficient intra-tile and inter-tile data movement and provides efficient computation support for generally inefficient bit-serial compute patterns. Our architecture consists of a massive hierarchical array of compute-enabled SRAMs (CRAMs), which is codesigned with a compiler to achieve high utilization. The key novelties of our architecture are (1) in providing efficient support for spatially aware communication by providing local H-tree network for reductions, by adding explicit hardware for shuffling operands, and by deploying systolic broadcasting, as well as (2) by taking advantage of the divisible nature of bit-serial computations through adaptive precision and efficient handling of constant operations. These innovations are integrated into a tensor expressions-based programming framework (including a compiler for easy programmability) that enables simple programmer control of optimizations for mapping programs into massively parallel binaries for millions of PIM processing elements. When compared against a similarly provisioned modern Tensor Core GPU (NVIDIA A100), across common DL kernels and end-to-end DL networks (Resnet18 and BERT), PIMSAB outperforms the GPU by 4.80\u00d7, and reduces energy by 3.76\u00d7. We compare PIMSAB with similarly provisioned state-of-the-art SRAM PIM (Duality Cache) and DRAM PIM (SIMDRAM), and observe a speedup of 3.7\u00d7 and 3.88\u00d7, respectively.<\/jats:p>","DOI":"10.1145\/3690824","type":"journal-article","created":{"date-parts":[[2024,9,5]],"date-time":"2024-09-05T16:09:37Z","timestamp":1725552577000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["PIMSAB: A Processing-In-Memory System with Spatially-Aware Communication and Bit-Serial-Aware Computation"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-0425-3335","authenticated-orcid":false,"given":"Siyuan","family":"Ma","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-1256-1528","authenticated-orcid":false,"given":"Kaustubh","family":"Mhatre","sequence":"additional","affiliation":[{"name":"Arizona State University, Tempe, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7933-9941","authenticated-orcid":false,"given":"Jian","family":"Weng","sequence":"additional","affiliation":[{"name":"King Abdullah University of Science and Technology, Thuwal, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8485-581X","authenticated-orcid":false,"given":"Bagus","family":"Hanindhito","sequence":"additional","affiliation":[{"name":"The University of Texas at Austin, Austin, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2366-4267","authenticated-orcid":false,"given":"Zhengrong","family":"Wang","sequence":"additional","affiliation":[{"name":"University of California, Los Angeles, Los Angeles, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8483-3824","authenticated-orcid":false,"given":"Tony","family":"Nowatzki","sequence":"additional","affiliation":[{"name":"University of California, Los Angeles, Los Angeles, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8747-5214","authenticated-orcid":false,"given":"Lizy","family":"John","sequence":"additional","affiliation":[{"name":"The University of Texas at Austin, Austin, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2547-4424","authenticated-orcid":false,"given":"Aman","family":"Arora","sequence":"additional","affiliation":[{"name":"Arizona State University, Tempe, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,11,20]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2017.21"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCD.1994.331954"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCAS45731.2020.9181068"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3297858.3304049"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/FCCM53951.2022.9786179"},{"key":"e_1_3_1_7_2","unstructured":"H. B. Bakoglu. 1990. Circuits interconnections and packaging for VLSI. Addison-Wesley Pub. Co."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSSC.2018.2880918"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4613-1619-0_18"},{"key":"e_1_3_1_10_2","volume-title":"13th OSDI","author":"Chen Tianqi","year":"2018","unstructured":"Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. TVM: An automated end-to-end optimizing compiler for deep learning. In 13th OSDI."},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2016.13"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2018.00040"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","unstructured":"Young-Cheon Kwon Suk Han Lee Jaehoon Lee Sang-Hyuk Kwon Je Min Ryu Jong-Pil Son O. Seongil Hak-Soo Yu Haesuk Lee Soo Young Kim Youngmin Cho Jin Guk Kim Jongyoon Choi Hyun-Sung Shin Jin Kim BengSeng Phuah HyoungMin Kim Myeong Jun Song Ahn Choi Daeho Kim SooYoung Kim Eun-Bong Kim David Wang Shinhaeng Kang Yuhwan Ro Seungwoo Seo JoonHo Song Jaeyoun Youn Kyomin Sohn and Nam Sung Kim. 2021. 25.4 A 20nm 6GB function-in-memory DRAM based on HBM2 with a 1.2TFLOPS programmable computing unit using bank-level parallelism for machine learning applications. In 2021 IEEE International Solid- State Circuits Conference (ISSCC). 350\u2013352. DOI:10.1109\/ISSCC42613.2021.9365862","DOI":"10.1109\/ISSCC42613.2021.9365862"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3173162.3173171"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3307650.3322257"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSCC42614.2022.9731754"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3352460.3358260"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/2966986.2980098"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3445814.3446749"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3472456.3472512"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","unstructured":"Chun Hok Ho Chi Wai Yu Philip H. W. Leong Wayne Luk and Steven J. E. Wilton. 2007. Domain-specific hybrid FPGA: Architecture and floating point applications. In 2007 International Conference on Field Programmable Logic and Applications. 196\u2013201. DOI:10.1109\/FPL.2007.4380647","DOI":"10.1109\/FPL.2007.4380647"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3307650.3322237"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVLSI.2017.2776954"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO.2016.7783722"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSSC.2017.2782087"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2020.3034117"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2014.6855225"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","unstructured":"Mingu Kang Min-Sun Keel Naresh R. Shanbhag Sean Eilert and Ken Curewitz. 2014. An energy-efficient VLSI architecture for pattern recognition via deep embedding of computation in SRAM. In 2014 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP). 8326\u20138330. DOI:10.1109\/ICASSP.2014.6855225","DOI":"10.1109\/ICASSP.2014.6855225"},{"key":"e_1_3_1_29_2","volume-title":"CUTLASS","author":"Kerr Andrew","year":"2022","unstructured":"Andrew Kerr, Haicheng Wu, Manish Gupta, Dustyn Blasig, Pradeep Ramini, Duane Merrill, Aniket Shivam, Piotr Majcher, Paul Springer, Markus Hohnerbach, Jin Wang, and Matt Nicely. 2022. CUTLASS. DOI:https:\/\/github.com\/NVIDIA\/cutlass"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3123939.3123977"},{"key":"e_1_3_1_31_2","volume-title":"Neural Information Processing Systems","author":"Murray Alan","year":"1987","unstructured":"Alan Murray, Anthony Smith, and Zoe Butler. 1987. Bit-serial neural networks. In Neural Information Processing Systems, D. Anderson (Ed.), Vol. 0. American Institute of Physics."},{"key":"e_1_3_1_32_2","volume-title":"FreePDK45","year":"2018","unstructured":"NCSU. 2018. FreePDK45. DOI:https:\/\/www.eda.ncsu.edu\/wiki\/FreePDK45:Contents"},{"key":"e_1_3_1_33_2","first-page":"117","volume-title":"2021 USENIX Annual Technical Conference (USENIX ATC 21)","author":"Nider Joel","year":"2021","unstructured":"Joel Nider, Craig Mustard, Andrada Zoltan, John Ramsden, Larry Liu, Jacob Grossbard, Mohammad Dashti, Romaric Jodin, Alexandre Ghiti, Jordi Chauzi, and Alexandra Fedorova. 2021. A case study of processing-in-memory in off-the-shelf systems. In 2021 USENIX Annual Technical Conference (USENIX ATC 21). USENIX Association, 117\u2013130. DOI:https:\/\/www.usenix.org\/conference\/atc21\/presentation\/nider"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/SOCC.2018.8618491"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA56546.2023.10071070"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185528"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3123939.3124544"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2016.12"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.vlsi.2017.02.002"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/296399.296442"},{"key":"e_1_3_1_41_2","volume-title":"Predictive Technology Model","author":"University Arizona State","year":"2012","unstructured":"Arizona State University. 2012. Predictive Technology Model. DOI:http:\/\/ptm.asu.edu\/"},{"key":"e_1_3_1_42_2","article-title":"Tensor comprehensions: Framework-agnostic high-performance machine learning abstractions","author":"Vasilache Nicolas","year":"2018","unstructured":"Nicolas Vasilache, Oleksandr Zinenko, Theodoros Theodoridis, Priya Goyal, Zachary DeVito, William S. Moses, Sven Verdoolaege, Andrew Adams, and Albert Cohen. 2018. Tensor comprehensions: Framework-agnostic high-performance machine learning abstractions. arXiv preprint arXiv:1802.04730 (2018).","journal-title":"arXiv preprint arXiv:1802.04730"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSSC.2019.2939682"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/FCCM51124.2021.00018"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/216585.216588"},{"key":"e_1_3_1_46_2","unstructured":"Pavan Yalamanchili Umar Arshad Zakiuddin Mohammed Pradeep Garigipati Peter Entschev Brian Kloppenborg James Malcolm and John Melonakos. 2015. ArrayFire - A high performance software library for parallel computing with an easy-to-use API. DOI:https:\/\/github.com\/arrayfire\/arrayfire"}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3690824","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3690824","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:58:06Z","timestamp":1750294686000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3690824"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,20]]},"references-count":45,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,12,31]]}},"alternative-id":["10.1145\/3690824"],"URL":"https:\/\/doi.org\/10.1145\/3690824","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"value":"1544-3566","type":"print"},{"value":"1544-3973","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,20]]},"assertion":[{"value":"2023-12-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-13","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}