{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,30]],"date-time":"2026-08-30T02:52:36Z","timestamp":1788058356838,"version":"build-2784847793"},"reference-count":130,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"name":"University of Wisconsin-Madison by the Vilas Life Cycle Professorship"},{"name":"Fall Research Competition"},{"name":"National Science Foundation","award":["ENS-1925485 and AAL5255"],"award-info":[{"award-number":["ENS-1925485 and AAL5255"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:p>\n                    Modern accelerators like GPUs increasingly execute independent operations concurrently to improve the device\u2019s compute utilization. However, effectively harnessing it on GPUs for important primitives such as general matrix multiplications (GEMMs) remains challenging. Although modern GPUs have significant hardware and software GEMM support, their kernel implementations and optimizations typically assume each kernel executes in\n                    <jats:italic toggle=\"yes\">isolation<\/jats:italic>\n                    and can utilize all GPU resources. This approach is highly efficient when kernels execute in isolation, but causes significant resource contention and slowdowns when kernels execute concurrently. Moreover, current approaches often only\n                    <jats:italic toggle=\"yes\">statically<\/jats:italic>\n                    expose and control parallelism within an application, without considering runtime information such as varying input size and concurrent applications\u2014often exacerbating contention. These issues limit performance benefits from concurrently executing independent operations. Accordingly, we propose GOLDYLOC , which considers the\n                    <jats:italic toggle=\"yes\">global<\/jats:italic>\n                    resources across all concurrent operations to identify performant GEMM kernels, which we call globally optimized (GO)-Kernels. GOLDYLOC also introduces a lightweight dynamic logic which considers the\n                    <jats:italic toggle=\"yes\">dynamic<\/jats:italic>\n                    execution environment for available parallelism and input sizes to execute performant combinations of concurrent GEMMs on the GPU. Overall, GOLDYLOC improves the performance of concurrent GEMMs on a real GPU by up to 2\u00d7 (18% geomean per workload) versus the default concurrency approach and provides up to 2.5\u00d7 (43% geomean per workload) speedup over sequential execution.\n                  <\/jats:p>","DOI":"10.1145\/3730584","type":"journal-article","created":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T07:08:38Z","timestamp":1746688118000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["GOLDYLOC: Global Optimizations &amp; Lightweight Dynamic Logic for Concurrency"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-1083-1146","authenticated-orcid":false,"given":"Suchita","family":"Pati","sequence":"first","affiliation":[{"name":"Research & Advanced Development, Advanced Micro Devices Inc","place":["Santa Clara, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9552-0508","authenticated-orcid":false,"given":"Shaizeen","family":"Aga","sequence":"additional","affiliation":[{"name":"Research & Advanced Development, Advanced Micro Devices Inc","place":["Santa Clara, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-2973-9479","authenticated-orcid":false,"given":"Nuwan","family":"Jayasena","sequence":"additional","affiliation":[{"name":"Research & Advanced Development, Advanced Micro Devices Inc","place":["Santa Clara, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0189-7895","authenticated-orcid":false,"given":"Matthew","family":"Sinclair","sequence":"additional","affiliation":[{"name":"Department of Computer Sciences, University of Wisconsin Madison","place":["Madison, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,7]]},"reference":[{"key":"e_1_3_3_2_2","first-page":"21","volume-title":"ISC HPC","author":"Abdelfattah Ahmad","year":"2016","unstructured":"Ahmad Abdelfattah, Azzam Haidar, Stanimire Tomov, and Jack Dongarra. 2016. Performance, design, and autotuning of batched GEMM for GPUs. In ISC HPC. 21\u201338."},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2012.6168946"},{"key":"e_1_3_3_4_2","unstructured":"AMD. 2018. HIP: Heterogeneous-computing Interface for Portability. Retrieved from https:\/\/github.com\/ROCm-Developer-Tools\/HIP\/"},{"key":"e_1_3_3_5_2","unstructured":"AMD. 2019. AMD Ryzen\u2122 Threadripper 2950X Processor. Retrieved from https:\/\/www.amd.com\/en\/products\/cpu\/amd-ryzen-threadripper-2950x"},{"key":"e_1_3_3_6_2","unstructured":"AMD. 2019. AMD\u2019s BLAS Library. Retrieved from https:\/\/github.com\/ROCmSoftwarePlatform\/rocBLAS"},{"key":"e_1_3_3_7_2","unstructured":"AMD. 2020. AMD CDNA Architecture. Retrieved from https:\/\/www.amd.com\/system\/files\/documents\/amd-cdna-whitepaper.pdf"},{"key":"e_1_3_3_8_2","unstructured":"AMD. 2020. AMD Instinct\u2122 MI100 Accelerator. Retrieved from https:\/\/www.amd.com\/en\/products\/server-accelerators\/instinct-mi100"},{"key":"e_1_3_3_9_2","unstructured":"AMD. 2020. AMD MxGPU and VMware. Retrieved from https:\/\/drivers.amd.com\/relnotes\/amd_mxgpu_deploymentguide_vmware.pdf"},{"key":"e_1_3_3_10_2","unstructured":"AMD. 2020. AMD\u2019s Tool for Creating a Benchmark-driven Backend Library for GEMMs. Retrieved from https:\/\/github.com\/ROCmSoftwarePlatform\/Tensile\/"},{"key":"e_1_3_3_11_2","unstructured":"AMD. 2022. AMD Instinct\u2122 MI210 Accelerator. Retrieved from https:\/\/www.amd.com\/en\/products\/accelerators\/instinct\/mi200\/mi210.html"},{"key":"e_1_3_3_12_2","unstructured":"AMD. 2024. AMD HSA Code Object Format. Retrieved from https:\/\/rocmdocs.amd.com\/en\/latest\/ROCm_Compiler_SDK\/ROCm-Codeobj-format.html"},{"key":"e_1_3_3_13_2","unstructured":"AMD. 2024. AMD Instinct\u2122 MI325X Accelerators. Retrieved from https:\/\/www.amd.com\/en\/products\/accelerators\/instinct\/mi300\/mi325x.html"},{"key":"e_1_3_3_14_2","unstructured":"AMD. 2024. AMD ROCm Profiler. Retrieved from https:\/\/rocmdocs.amd.com\/en\/latest\/ROCm_Tools\/ROCm-Tools.html"},{"key":"e_1_3_3_15_2","unstructured":"AMD. 2024. Use ROCm on Radeon GPUs Documentation. Retrieved from https:\/\/rocm.docs.amd.com\/_\/downloads\/radeon\/en\/latest\/pdf\/"},{"key":"e_1_3_3_16_2","first-page":"173","volume-title":"ICML","year":"2016","unstructured":"Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Qiang Cheng, Guoliang Chen, Jie Chen, Jingdong Chen, Zhijie Chen, Mike Chrzanowski, Adam Coates, Greg Diamos, Ke Ding, Niandong Du, Erich Elsen, Jesse Engel, Weiwei Fang, Linxi Fan, Christopher Fougner, Liang Gao, Caixia Gong, Awni Hannun, Tony Han, Lappi Johannes, Bing Jiang, Cai Ju, Billy Jun, Patrick LeGresley, Libby Lin, Junjie Liu, Yang Liu, Weigao Li, Xiangang Li, Dongpeng Ma, Sharan Narang, Andrew Ng, Sherjil Ozair, Yiping Peng, Ryan Prenger, Sheng Qian, Zongfeng Quan, Jonathan Raiman, Vinay Rao, Sanjeev Satheesh, David Seetapun, Shubho Sengupta, Kavya Srinet, Anuroop Sriram, Haiyuan Tang, Liliang Tang, Chong Wang, Jidong Wang, Kaifu Wang, Yi Wang, Zhijian Wang, Zhiqian Wang, Shuang Wu, Likai Wei, Bo Xiao, Wen Xie, Yan Xie, Dani Yogatama, Bin Yuan, Jun Zhan, and Zhenyao Zhu. 2016. Deep speech 2: End-to-End speech recognition in English and Mandarin. In ICML. 173\u2013182."},{"key":"e_1_3_3_17_2","unstructured":"Jeremy Appleyard Tomas Kocisky and Phil Blunsom. 2016. Optimizing performance of recurrent neural networks on GPUs. CoRR abs\/1604.01946 (2016). Retrieved from http:\/\/arxiv.org\/abs\/1604.01946"},{"key":"e_1_3_3_18_2","unstructured":"Ashraf Eassa and Sukru Burc Eryilmaz NVIDIA. 2022. The full stack optimization powering NVIDIA MLPerf Training v2.0 Performance. Retrieved from https:\/\/developer.nvidia.com\/blog\/boosting-mlperf-training-performance-with-full-stack-optimization\/"},{"key":"e_1_3_3_19_2","unstructured":"Jimmy Lei Ba Jamie Ryan Kiros and Geoffrey E. Hinton. 2016. Layer normalization. CoRR abs\/1607.06450 (2016). Retrieved from https:\/\/arxiv.org\/abs\/1607.06450"},{"key":"e_1_3_3_20_2","volume-title":"ICLR","author":"Bahdanau Dzmitry","year":"2015","unstructured":"Dzmitry Bahdanau, KyungHyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR."},{"key":"e_1_3_3_21_2","first-page":"1877","volume-title":"NeurIPS","year":"2020","unstructured":"Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In NeurIPS, Vol. 33. 1877\u20131901."},{"issue":"3","key":"e_1_3_3_22_2","doi-asserted-by":"crossref","first-page":"693","DOI":"10.1111\/j.2517-6161.1993.tb01934.x","article-title":"On robustness in the logistic regression model","volume":"55","author":"Carroll Raymond J.","year":"1993","unstructured":"Raymond J. Carroll and Shane Pederson. 1993. On robustness in the logistic regression model. Journal of the Royal Statistical Society: Series B (Methodological) 55, 3 (1993), 693\u2013706.","journal-title":"Journal of the Royal Statistical Society: Series B (Methodological)"},{"key":"e_1_3_3_23_2","first-page":"17","volume-title":"ASPLOS","author":"Chen Quan","year":"2017","unstructured":"Quan Chen, Hailong Yang, Minyi Guo, Ram Srivatsa Kannan, Jason Mars, and Lingjia Tang. 2017. Prophet: Precise QoS prediction on non-preemptive accelerators to improve utilization in warehouse-scale computers. In ASPLOS. 17\u201332."},{"key":"e_1_3_3_24_2","first-page":"681","volume-title":"ASPLOS","author":"Chen Quan","year":"2016","unstructured":"Quan Chen, Hailong Yang, Jason Mars, and Lingjia Tang. 2016. Baymax: QoS awareness and increased utilization for non-preemptive accelerators in warehouse scale computers. In ASPLOS. 681\u2013696."},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1179"},{"key":"e_1_3_3_26_2","first-page":"493","volume-title":"HPCA","author":"Choi Yujeong","year":"2021","unstructured":"Yujeong Choi, Yunseong Kim, and Minsoo Rhu. 2021. Lazy batching: An SLA-aware batching system for cloud machine learning inference. In HPCA. 493\u2013506."},{"key":"e_1_3_3_27_2","first-page":"220","volume-title":"HPCA","author":"Choi Yujeong","year":"2020","unstructured":"Yujeong Choi and Minsoo Rhu. 2020. PREMA: A predictive multi-task scheduling algorithm for preemptible neural processing units. In HPCA. 220\u2013233."},{"key":"e_1_3_3_28_2","first-page":"20","volume-title":"EMNLP-IJCNP","author":"Correia Gon\u00e7alo M.","year":"2019","unstructured":"Gon\u00e7alo M. Correia, Vlad Niculae, and Andr\u00e9 F. T. Martins. 2019. Adaptively sparse transformers. In EMNLP-IJCNP. 20 pages."},{"key":"e_1_3_3_29_2","first-page":"700","volume-title":"MICRO","author":"Dalmia Preyesh","year":"2024","unstructured":"Preyesh Dalmia, Rajesh Shashi Kumar, and Matthew D. Sinclair. 2024. CPElide: Efficient multi-chiplet GPU implicit synchronization. In MICRO. 700\u2013717."},{"key":"e_1_3_3_30_2","unstructured":"Dell Technologies. 2022. MLPerf\u2122 v1.1 Inference on Virtualized and Multi-Instance GPUs."},{"key":"e_1_3_3_31_2","first-page":"4171","volume-title":"NAACL-HLT","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT. 4171\u20134186."},{"key":"e_1_3_3_32_2","first-page":"1","volume-title":"MICRO","author":"Hajj Izzat El","year":"2016","unstructured":"Izzat El Hajj, Juan Gomez-Luna, Cheng Li, Li-Wen Chang, Dejan Milojicic, and Wen-mei Hwu. 2016. KLAP: Kernel launch aggregation and promotion for optimizing dynamic parallelism. In MICRO. 1\u201312."},{"key":"e_1_3_3_33_2","first-page":"33","volume-title":"RTSS","author":"Elliott Glenn A.","year":"2013","unstructured":"Glenn A. Elliott, Bryan C. Ward, and James H. Anderson. 2013. GPUSync: A framework for real-time GPU management. In RTSS. 33\u201344."},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11227-015-1483-z"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/2082156.2082183"},{"key":"e_1_3_3_36_2","first-page":"407","volume-title":"MICRO","author":"Fung Wilson W. L.","year":"2007","unstructured":"Wilson W. L. Fung, Ivan Sham, George Yuan, and Tor M. Aamodt. 2007. Dynamic warp formation and scheduling for efficient GPU control flow. In MICRO. 407\u2013420."},{"key":"e_1_3_3_37_2","volume-title":"EuroSys","author":"Gao Pin","year":"2018","unstructured":"Pin Gao, Lingfan Yu, Yongwei Wu, and Jinyang Li. 2018. Low latency RNN inference with cellular batching. In EuroSys. Article 31, 15 pages."},{"issue":"03","key":"e_1_3_3_38_2","article-title":"AI and memory wall","volume":"44","author":"Gholami Amir","year":"2024","unstructured":"Amir Gholami, Zhewei Yao, Sehoon Kim, Coleman Hooper, Michael W. Mahoney, and Kurt Keutzer. 2024. AI and memory wall. IEEE Micro 44, 03 (2024).","journal-title":"IEEE Micro"},{"key":"e_1_3_3_39_2","first-page":"982","volume-title":"ISCA","author":"Gupta Udit","year":"2020","unstructured":"Udit Gupta, Samuel Hsia, Vikram Saraph, Xiaodong Wang, Brandon Reagen, Gu-Yeon Wei, Hsien-Hsin S. Lee, David Brooks, and Carole-Jean Wu. 2020. DeepRecSys: A system for optimizing end-to-end at-scale neural recommendation inference. In ISCA. 982\u2013995."},{"key":"e_1_3_3_40_2","first-page":"608","volume-title":"HPCA","year":"2018","unstructured":"Anthony Gutierrez, Bradford M. Beckmann, Alexandru Dutu, Joseph Gross, Michael LeBeane, John Kalamatianos, Onur Kayiran, Matthew Poremba, Brandon Potter, Sooraj Puthoor, Matthew D. Sinclair, Mark Wyse, Jieming Yin, Xianwei Zhang, Akshay Jain, and Timothy Rogers. 2018. Lost in abstraction: Pitfalls of analyzing GPUs at the intermediate language level. In HPCA. 608\u2013619."},{"key":"e_1_3_3_41_2","first-page":"81","volume-title":"2nd International Workshop on Parallel Algorithms for Irregularly Structured Problems","author":"Hao Benjamin","year":"1995","unstructured":"Benjamin Hao and David Pearson. 1995. Instruction scheduling and global register allocation for SIMD multiprocessors. In 2nd International Workshop on Parallel Algorithms for Irregularly Structured Problems. Springer, Berlin, 81\u201386."},{"key":"e_1_3_3_42_2","first-page":"372","volume-title":"MICRO","author":"He Mingxuan","year":"2020","unstructured":"Mingxuan He, Choungki Song, Ilkon Kim, Chunseok Jeong, Seho Kim, Il Park, Mithuna Thottethodi, and TN Vijaykumar. 2020. Newton: A DRAM-maker\u2019s accelerator-in-memory (AiM) architecture for machine learning. In MICRO. 372\u2013385."},{"key":"e_1_3_3_43_2","unstructured":"Yanzhang He Tara N. Sainath Rohit Prabhavalkar Ian McGraw Raziel Alvarez Ding Zhao David Rybach Anjuli Kannan Yonghui Wu Ruoming Pang et\u00a0al. 2018. Streaming end-to-end speech recognition for mobile devices. CoRR abs\/1811.06621 (2018). Retrieved from https:\/\/arxiv.org\/abs\/1811.06621"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_3_45_2","first-page":"41:1\u201341:16","volume-title":"EuroSys","author":"Holmes Connor","year":"2019","unstructured":"Connor Holmes, Daniel Mawhirter, Yuxiong He, Feng Yan, and Bo Wu. 2019. GRNN: Low-latency and scalable RNN inference on GPUs. In EuroSys. 41:1\u201341:16."},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-0258(19970515)16:9<965::AID-SIM509>3.0.CO;2-O"},{"key":"e_1_3_3_47_2","first-page":"14","volume-title":"ISCA","author":"Hwang Ranggi","year":"2024","unstructured":"Ranggi Hwang, Jianyu Wei, Shijie Cao, Changho Hwang, Xiaohu Tang, Ting Cao, and Mao Yang. 2024. Pre-gated MoE: An algorithm-system co-design for fast and scalable mixture-of-expert inference. In ISCA. 14 pages."},{"key":"e_1_3_3_48_2","first-page":"711","volume-title":"Proceedings of Machine Learning and Systems","volume":"3","author":"Ivanov Andrei","year":"2020","unstructured":"Andrei Ivanov, Nikoli Dryden, Tal Ben-Nun, Shigang Li, and Torsten Hoefler. 2020. Data movement is all you need: A case study on optimizing transformers. In Proceedings of Machine Learning and Systems. A. Smola, A. Dimakis, and I. Stoica (Eds.), Vol. 3. 711\u2013732."},{"key":"e_1_3_3_49_2","first-page":"163","volume-title":"ISCA","author":"Jablin James A.","year":"2014","unstructured":"James A. Jablin, Thomas B. Jablin, Onur Mutlu, and Maurice Herlihy. 2014. Warp-aware trace scheduling for GPUs. In ISCA. 163\u2013174."},{"key":"e_1_3_3_50_2","first-page":"8","volume-title":"NeurIPS","author":"Jain Paras","year":"2018","unstructured":"Paras Jain, Xiangxi Mo, Ajay Jain, Harikaran Subbaraj, Rehan Sohail Durrani, Alexey Tumanov, Joseph Gonzalez, and Ion Stoica. 2018. Dynamic space-time scheduling for GPU inference. In NeurIPS. 8 pages."},{"key":"e_1_3_3_51_2","first-page":"249","volume-title":"ASPLOS","author":"Jaleel Aamer","year":"2012","unstructured":"Aamer Jaleel, Hashem H. Najaf-abadi, Samantika Subramaniam, Simon C. Steely, and Joel Emer. 2012. CRUISE: Cache replacement and utility-aware scheduling. In ASPLOS. 249\u2013260."},{"key":"e_1_3_3_52_2","first-page":"2","volume-title":"4th gem5 Users\u2019 Workshop","author":"Jamieson Charles","year":"2022","unstructured":"Charles Jamieson, Anushka Chandrashekar, Ian McDougall, and Matthew D. Sinclair. 2022. GAP: gem5 GPU accuracy profiler. In 4th gem5 Users\u2019 Workshop. 2 pages."},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2014.09.003"},{"key":"e_1_3_3_54_2","unstructured":"Zhihao Jia Matei Zaharia and Alex Aiken. 2018. Beyond data and model parallelism for deep neural networks. CoRR abs\/1807.05358 (2018). Retrieved from https:\/\/arxiv.org\/abs\/1807.05358"},{"key":"e_1_3_3_55_2","first-page":"395","volume-title":"ASPLOS","author":"Jog Adwait","year":"2013","unstructured":"Adwait Jog, Onur Kayiran, Nachiappan Chidambaram Nachiappan, Asit K. Mishra, Mahmut T. Kandemir, Onur Mutlu, Ravishankar Iyer, and Chita R. Das. 2013. OWL: Cooperative thread array aware scheduling techniques for improving GPGPU performance. In ASPLOS. 395\u2013406."},{"key":"e_1_3_3_56_2","first-page":"1","volume-title":"ISCA","year":"2021","unstructured":"Norman P. Jouppi, Doe Hyun Yoon, Matthew Ashcraft, Mark Gottscho, Thomas B. Jablin, George Kurian, James Laudon, Sheng Li, Peter Ma, Xiaoyu Ma, Thomas Norrie, Nishant Patil, Sushma Prasad, Cliff Young, Zongwei Zhou, and David Patterson. 2021. Ten lessons from three generations shaped Google\u2019s TPUv4i. In ISCA. 1\u201314."},{"key":"e_1_3_3_57_2","first-page":"1","volume-title":"ISCA","year":"2017","unstructured":"Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, Rick Boyle, Pierre-luc Cantin, Clifford Chao, Chris Clark, Jeremy Coriell, Mike Daley, Matt Dau, Jeffrey Dean, Ben Gelb, Tara Vazir Ghaemmaghami, Rajendra Gottipati, William Gulland, Robert Hagmann, C. Richard Ho, Doug Hogberg, John Hu, Robert Hundt, Dan Hurt, Julian Ibarz, Aaron Jaffey, Alek Jaworski, Alexander Kaplan, Harshit Khaitan, Daniel Killebrew, Andy Koch, Naveen Kumar, Steve Lacy, James Laudon, James Law, Diemthu Le, Chris Leary, Zhuyuan Liu, Kyle Lucke, Alan Lundin, Gordon MacKean, Adriana Maggiore, Maire Mahony, Kieran Miller, Rahul Nagarajan, Ravi Narayanaswami, Ray Ni, Kathy Nix, Thomas Norrie, Mark Omernick, Narayana Penukonda, Andy Phelps, Jonathan Ross, Matt Ross, Amir Salek, Emad Samadiani, Chris Severn, Gregory Sizikov, Matthew Snelham, Jed Souter, Dan Steinberg, Andy Swing, Mercedes Tan, Gregory Thorson, Bo Tian, Horia Toma, Erick Tuttle, Vijay Vasudevan, Richard Walter, Walter Wang, Eric Wilcox, and Doe Hyun Yoon. 2017. In-datacenter performance analysis of a tensor processing unit. In ISCA. 1\u201312."},{"key":"e_1_3_3_58_2","first-page":"814","volume-title":"HPCA","author":"Kao Sheng-Chun","year":"2022","unstructured":"Sheng-Chun Kao and Tushar Krishna. 2022. MAGMA: An optimization framework for mapping multiple DNNs on multiple accelerator cores. In HPCA. 814\u2013830."},{"key":"e_1_3_3_59_2","first-page":"14","volume-title":"USENIX ATC","author":"Kato Shinpei","year":"2011","unstructured":"Shinpei Kato, Karthik Lakshmanan, Ragunathan Rajkumar, and Yutaka Ishikawa. 2011. TimeGraph: GPU scheduling for real-time multi-tasking environments. In USENIX ATC. 14 pages."},{"key":"e_1_3_3_60_2","unstructured":"Jehandad Khan Paul Fultz Artem Tamazov Daniel Lowell Chao Liu Michael Melesse Murali Nandhimandalam Kamil Nasyrov Ilya Perminov Tejash Shah et\u00a0al. 2019. MIOpen: An open source library for deep learning primitives. CoRR abs\/1910.00078 (2019). Retrieved from https:\/\/arxiv.org\/abs\/1910.00078"},{"key":"e_1_3_3_61_2","first-page":"96","volume-title":"ICS","author":"Kim Jinsung","year":"2018","unstructured":"Jinsung Kim, Aravind Sukumaran-Rajam, Changwan Hong, Ajay Panyala, Rohit Kumar Srivastava, Sriram Krishnamoorthy, and P. Sadayappan. 2018. Optimizing tensor Contractions in CCSD(T) for efficient execution on GPUs. In ICS. 96\u2013106."},{"key":"e_1_3_3_62_2","first-page":"85","volume-title":"CGO","author":"Kim Jinsung","year":"2019","unstructured":"Jinsung Kim, Aravind Sukumaran-Rajam, Vineeth Thumma, Sriram Krishnamoorthy, Ajay Panyala, Louis-No\u00ebl Pouchet, Atanas Rountev, and P. Sadayappan. 2019. A code generator for high-performance tensor contractions on GPUs. In CGO. 85\u201395."},{"key":"e_1_3_3_63_2","first-page":"1169","volume-title":"MICRO","author":"Kotra Jagadish B.","year":"2021","unstructured":"Jagadish B. Kotra, Michael LeBeane, Mahmut T. Kandemir, and Gabriel H. Loh. 2021. Increasing GPU translation reach by leveraging under-utilized on-chip resources. In MICRO. 1169\u20131181."},{"key":"e_1_3_3_64_2","unstructured":"Hyoukjun Kwon Liangzhen Lai Tushar Krishna and Vikas Chandra. 2019. Herald: Optimizing heterogeneous DNN accelerators for edge devices. CoRR abs\/1909.07437 (2019). Retrieved from https:\/\/arxiv.org\/abs\/1909.07437"},{"key":"e_1_3_3_65_2","volume-title":"GPGPU","author":"Lakshminarayana Nagesh B.","year":"2010","unstructured":"Nagesh B. Lakshminarayana and Hyesoon Kim. 2010. Effect of instruction fetch and memory scheduling on GPU performance. In GPGPU, Vol. 88."},{"key":"e_1_3_3_66_2","volume-title":"PACT","author":"LeBeane Michael","year":"2018","unstructured":"Michael LeBeane, Khaled Hamidouche, Brad Benton, Mauricio Breternitz, Steven K. Reinhardt, and Lizy K. John. 2018. ComP-Net: Command processor networking for efficient intra-kernel communications on GPUs. In PACT (Limassol, Cyprus). Article 29, 13 pages."},{"key":"e_1_3_3_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2016.79"},{"key":"e_1_3_3_68_2","first-page":"515","volume-title":"ISCA","author":"Lee Shin-Ying","year":"2015","unstructured":"Shin-Ying Lee, Akhil Arunkumar, and Carole-Jean Wu. 2015. CAWA: Coordinated warp scheduling and cache prioritization for critical warp acceleration of GPGPU workloads. In ISCA. 515\u2013527."},{"key":"e_1_3_3_69_2","first-page":"14","volume-title":"CGO","author":"Li Ao","year":"2022","unstructured":"Ao Li, Bojian Zheng, Gennady Pekhimenko, and Fan Long. 2022. Automatic horizontal fusion for GPU kernels. In CGO. 14\u201327."},{"key":"e_1_3_3_70_2","first-page":"506","volume-title":"ICPADS","author":"Liu Jie","year":"2019","unstructured":"Jie Liu, Jiawen Liu, Wan Du, and Dong Li. 2019. Performance analysis and characterization of training deep learning models on mobile device. In ICPADS. 506\u2013515."},{"key":"e_1_3_3_71_2","first-page":"383","volume-title":"MICRO","author":"Liu Jiwei","year":"2015","unstructured":"Jiwei Liu, Jun Yang, and Rami Melhem. 2015. SAWS: Synchronization aware GPGPU warp scheduling for multiple independent warp schedulers. In MICRO. 383\u2013394."},{"key":"e_1_3_3_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3503222.3507752"},{"key":"e_1_3_3_73_2","unstructured":"Jason Lowe-Power Abdul Mutaal Ahmad Ayaz Akram Mohammad Alian Rico Amslinger Matteo Andreozzi Adria Armejach Nils Asmussen Srikant Bharadwaj Gabe Black et\u00a0al. 2020. The gem5 Simulator: Version 20.0+. CoRR abs\/2007.03152 (2020). Retrieved from https:\/\/arxiv.org\/abs\/2007.03152"},{"key":"e_1_3_3_74_2","unstructured":"Justin Luitjens. NVIDIA. 2014. CUDA streams: Best practices and common pitfalls. GPU Technology Conference (GTC) 2016. Retrieved from https:\/\/www.hpcadmintech.com\/wp-content\/uploads\/2016\/03\/Carlo_Nardone_presentation.pdf"},{"key":"e_1_3_3_75_2","first-page":"881","volume-title":"OSDI","author":"Ma Lingxiao","year":"2020","unstructured":"Lingxiao Ma, Zhiqiang Xie, Zhi Yang, Jilong Xue, Youshan Miao, Wei Cui, Wenxiang Hu, Fan Yang, Lintao Zhang, and Lidong Zhou. 2020. Rammer: Enabling holistic deep learning compiler optimizations with rtasks. In OSDI. 881\u2013897."},{"key":"e_1_3_3_76_2","unstructured":"Paulius Micikevicius Sharan Narang Jonah Alben Gregory Diamos Erich Elsen David Garcia Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh et\u00a0al. 2018. Mixed precision training. CoRR abs\/1710.03740 (2018). Retrieved from https:\/\/arxiv.org\/abs\/1710.03740"},{"issue":"8","key":"e_1_3_3_77_2","first-page":"8","article-title":"Turing-NLG: A 17-billion-parameter language model by Microsoft","volume":"1","year":"2020","unstructured":"Microsoft. 2020. Turing-NLG: A 17-billion-parameter language model by Microsoft. Microsoft Research Blog 1, 8 (2020), 8 pages.","journal-title":"Microsoft Research Blog"},{"key":"e_1_3_3_78_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA52012.2021.00014"},{"key":"e_1_3_3_79_2","unstructured":"Sharan Narang Gregory F. Diamos Shubho Sengupta and Erich Elsen. 2017. Exploring sparsity in recurrent neural networks. CoRR abs\/1704.05119 (2017). Retrieved from https:\/\/arxiv.org\/abs\/1704.05119"},{"key":"e_1_3_3_80_2","first-page":"308","volume-title":"MICRO","author":"Narasiman Veynu","year":"2011","unstructured":"Veynu Narasiman, Michael Shebanow, Chang Joo Lee, Rustam Miftakhutdinov, Onur Mutlu, and Yale N. Patt. 2011. Improving GPU performance via large warps and two-level warp scheduling. In MICRO. 308\u2013317."},{"key":"e_1_3_3_81_2","unstructured":"Nathan Benaich and Air Street Capital. 2022. State of AI Report 2022. Retrieved from https:\/\/www.stateof.ai\/"},{"key":"e_1_3_3_82_2","first-page":"969","volume-title":"ICPP","author":"Nelson Thomas","year":"2015","unstructured":"Thomas Nelson, Axel Rivera, Prasanna Balaprakash, Mary Hall, Paul D. Hovland, Elizabeth Jessup, and Boyana Norris. 2015. Generating efficient tensor contractions for GPUs. In ICPP. 969\u2013978."},{"key":"e_1_3_3_83_2","first-page":"15","volume-title":"4th RISC-V Workshop","year":"2016","unstructured":"NVIDIA. 2016. NVIDIA RISC-V Story. In 4th RISC-V Workshop. RISC-V, San Francisco, CA, 15 pages."},{"key":"e_1_3_3_84_2","unstructured":"NVIDIA. 2017. NVIDIA Tesla V100 GPU Architecture The World\u2019s Most Advanced Data Center GPU. Retrieved from https:\/\/images.nvidia.com\/content\/volta-architecture\/pdf\/volta-architecture-whitepaper.pdf"},{"key":"e_1_3_3_85_2","unstructured":"NVIDIA. 2017. Pro Tip: cuBLAS Strided Batched Matrix Multiply. Retrieved from https:\/\/developer.nvidia.com\/blog\/cublas-strided-batched-matrix-multiply\/"},{"key":"e_1_3_3_86_2","unstructured":"NVIDIA. 2018. CUDA Stream Management. Retrieved from https:\/\/docs.nvidia.com\/cuda\/cuda-c-programming-guide\/index.html#streams"},{"key":"e_1_3_3_87_2","unstructured":"NVIDIA. 2018. Megatron-LM Github. Retrieved from https:\/\/github.com\/NVIDIA\/Megatron-LM"},{"key":"e_1_3_3_88_2","unstructured":"Shankar Chandrasekaran NVIDIA. 2019. Easily Deploy Deep Learning Models in Production. Retrieved from https:\/\/medium.com\/dataseries\/easily-deploy-deep-learning-models-in-production-13db48071578"},{"key":"e_1_3_3_89_2","unstructured":"NVIDIA. 2019. NVIDIA Deep Learning Performance. Retrieved from https:\/\/docs.nvidia.com\/deeplearning\/performance\/index.html"},{"key":"e_1_3_3_90_2","unstructured":"NVIDIA. 2020. Ride the Fast Lane to AI Productivity with Multi-Instance GPUs. Retrieved from https:\/\/blogs.nvidia.com\/blog\/2020\/05\/14\/multi-instance-gpus\/"},{"key":"e_1_3_3_91_2","unstructured":"NVIDIA. 2024. NVIDIA cuDNN: GPU Accelerated Deep Learning. Retrieved from https:\/\/developer.nvidia.com\/cudnn"},{"key":"e_1_3_3_92_2","unstructured":"NVIDIA. 2024. NVIDIA H200 Tensor Core GPU. Retrieved from https:\/\/www.nvidia.com\/en-us\/data-center\/h200\/"},{"key":"e_1_3_3_93_2","unstructured":"NVIDIA Corp.2024. NVIDIA cuBLAS. Retrieved from https:\/\/developer.nvidia.com\/cublas"},{"key":"e_1_3_3_94_2","unstructured":"NVIDIA Corp.2024. NVIDIA Multi-Instance GPU (MIG). Retrieved from https:\/\/docs.nvidia.com\/cuda\/mig\/index.html"},{"key":"e_1_3_3_95_2","volume-title":"ECRTS","author":"Otterness Nathan","year":"2020","unstructured":"Nathan Otterness and James H. Anderson. 2020. AMD GPUs as an alternative to NVIDIA for supporting real-time workloads. In ECRTS, Vol. 165."},{"key":"e_1_3_3_96_2","volume-title":"RTNS","author":"Otterness Nathan","year":"2021","unstructured":"Nathan Otterness and James H. Anderson. 2021. Exploring AMD GPU scheduling details by experimenting with \u201cWorst Practices\u201d. In RTNS."},{"key":"e_1_3_3_97_2","first-page":"407","volume-title":"ASPLOS","author":"Pai Sreepathi","year":"2013","unstructured":"Sreepathi Pai, Matthew J. Thazhuthaveetil, and R. Govindarajan. 2013. Improving GPGPU concurrency with elastic kernels. In ASPLOS. 407\u2013418."},{"key":"e_1_3_3_98_2","volume-title":"Cross-Stack Optimizations for Sequence-Based Models on GPUs","author":"Pati Suchita","year":"2024","unstructured":"Suchita Pati. 2024. Cross-Stack Optimizations for Sequence-Based Models on GPUs. Ph. D. Dissertation. The University of Wisconsin-Madison."},{"key":"e_1_3_3_99_2","first-page":"14","volume-title":"IISWC","author":"Pati Suchita","year":"2022","unstructured":"Suchita Pati, Shaizeen Aga, Nuwan Jayasena, and Matthew D. Sinclair. 2022. Demystifying BERT: System design implications. In IISWC. 14 pages."},{"key":"e_1_3_3_100_2","first-page":"69","volume-title":"ISPASS","author":"Pati Suchita","year":"2020","unstructured":"Suchita Pati, Shaizeen Aga, Matthew D. Sinclair, and Nuwan Jayasena. 2020. SeqPoint: Identifying representative iterations of sequence-based neural networks. In ISPASS. 69\u201380."},{"key":"e_1_3_3_101_2","first-page":"50","volume-title":"GPGPU","author":"Puthoor Sooraj","year":"2018","unstructured":"Sooraj Puthoor, Xulong Tang, Joseph Gross, and Bradford M. Beckmann. 2018. Oversubscribed command queues in GPUs. In GPGPU. 50\u201360."},{"key":"e_1_3_3_102_2","unstructured":"PyTorch. 2019. Pytorch Automatic Mixed Precision Package. Retrieved from https:\/\/pytorch.org\/docs\/stable\/amp.html"},{"key":"e_1_3_3_103_2","first-page":"58","volume-title":"HPCA","author":"Qin Eric","year":"2020","unstructured":"Eric Qin, Ananda Samajdar, Hyoukjun Kwon, Vineet Nadella, Sudarshan Srinivasan, Dipankar Das, Bharat Kaul, and Tushar Krishna. 2020. Sigma: A sparse and irregular GEMM accelerator with flexible interconnects for DNN training. In HPCA. 58\u201370."},{"issue":"8","key":"e_1_3_3_104_2","first-page":"24","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 24 pages.","journal-title":"OpenAI Blog"},{"key":"e_1_3_3_105_2","first-page":"2","volume-title":"5th gem5 Users\u2019 Workshop","author":"Ramadas Vishnu","year":"2023","unstructured":"Vishnu Ramadas, Daniel Kouchekinia, Ndubuisi Osuji, and Matthew D. Sinclair. 2023. Closing the gap: Improving the accuracy of gem5\u2019s GPU models. In 5th gem5 Users\u2019 Workshop. 2 pages."},{"key":"e_1_3_3_106_2","first-page":"3","volume-title":"YArch","author":"Ramadas Vishnu","year":"2024","unstructured":"Vishnu Ramadas, Daniel Kouchekinia, and Matthew D. Sinclair. 2024. Further closing the GAP: Improving the accuracy of gem5\u2019s GPU models. In YArch. 3 pages."},{"key":"e_1_3_3_107_2","first-page":"144","volume-title":"EuroMLSys\u201924","author":"Recasens Pol G.","year":"2024","unstructured":"Pol G. Recasens, Yue Zhu, Chen Wang, Eun Kyung Lee, Olivier Tardieu, Alaa Youssef, Jordi Torres, and Josep Ll. Berral. 2024. Towards pareto optimal throughput in small language model serving. In EuroMLSys\u201924. 144\u2013152."},{"key":"e_1_3_3_108_2","first-page":"4","volume-title":"3rd gem5 Users\u2019 Workshop","author":"Roarty Kyle","year":"2020","unstructured":"Kyle Roarty and Matthew D. Sinclair. 2020. Modeling modern GPU applications in gem5. In 3rd gem5 Users\u2019 Workshop. 4 pages."},{"key":"e_1_3_3_109_2","first-page":"72","volume-title":"MICRO","author":"Rogers Timothy G.","year":"2012","unstructured":"Timothy G. Rogers, Mike O\u2019Connor, and Tor M. Aamodt. 2012. Cache-conscious wavefront scheduling. In MICRO. 72\u201383."},{"key":"e_1_3_3_110_2","first-page":"99","volume-title":"MICRO","author":"Rogers Timothy G.","year":"2013","unstructured":"Timothy G. Rogers, Mike O\u2019Connor, and Tor M. Aamodt. 2013. Divergence-aware warp scheduling. In MICRO. 99\u2013110."},{"key":"e_1_3_3_111_2","volume-title":"Learning representations by back-propagating errors","author":"Rumelhart David E.","year":"1986","unstructured":"David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. 1986. Learning representations by back-propagating errors. Nature 323, 6088 (1986), 533\u2013536."},{"key":"e_1_3_3_112_2","first-page":"193","volume-title":"HiPC","author":"Shi Yang","year":"2016","unstructured":"Yang Shi, U. N. Niranjan, Animashree Anandkumar, and Cris Cecka. 2016. Tensor contractions with extended BLAS kernels on CPU and GPU. In HiPC. 193\u2013202."},{"key":"e_1_3_3_113_2","unstructured":"Mohammad Shoeybi Mostofa Patwary Raul Puri Patrick LeGresley Jared Casper and Bryan Catanzaro. 2019. Megatron-LM: Training multi-billion parameter language models using model parallelism. CoRR abs\/1909.08053 (2019). Retrieved from https:\/\/arxiv.org\/abs\/1909.08053"},{"key":"e_1_3_3_114_2","unstructured":"Sklearn. 2019. Sklearn multi-class logistic regression. Retrieved from https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.linear_model.LogisticRegression.html"},{"key":"e_1_3_3_115_2","first-page":"48","volume-title":"ARRAY","author":"Springer Matthias","year":"2017","unstructured":"Matthias Springer, Peter Wauligmann, and Hidehiko Masuhara. 2017. Modular array-based GPU computing in a dynamically-typed language. In ARRAY. 48\u201355."},{"key":"e_1_3_3_116_2","first-page":"538","volume-title":"SC","year":"2022","unstructured":"Qingxiao Sun, Yi Liu, Hailong Yang, Ruizhe Zhang, Ming Dun, Mingzhen Li, Xiaoyan Liu, Wencong Xiao, Yong Liy, Zhongzhi Luan, Depei Qian. 2022. CoGNN: Efficient scheduling for concurrent GNN training on GPUs. In SC. 538\u2013552."},{"key":"e_1_3_3_117_2","volume-title":"GPUPool: A Holistic Approach to Fine-Grained GPU Sharing in the Cloud","author":"Tan Xiaodan","year":"2021","unstructured":"Xiaodan Tan. 2021. GPUPool: A Holistic Approach to Fine-Grained GPU Sharing in the Cloud. Ph. D. Dissertation. University of Toronto (Canada)."},{"key":"e_1_3_3_118_2","unstructured":"TIRIAS Research. 2019. Why Your AI Infrastructure Needs Both Training and Inference [White paper]."},{"key":"e_1_3_3_119_2","first-page":"6000","volume-title":"NeurIPS","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS. 6000\u20136010."},{"key":"e_1_3_3_120_2","first-page":"344","volume-title":"GREENCOM-CPSCOM","author":"Wang Guibin","year":"2010","unstructured":"Guibin Wang, YiSong Lin, and Wei Yi. 2010. Kernel fusion: An effective method for better power efficiency on multithreaded GPU. In GREENCOM-CPSCOM. 344\u2013350."},{"key":"e_1_3_3_121_2","first-page":"7686","volume-title":"NeurIPS","author":"Wang Naigang","year":"2018","unstructured":"Naigang Wang, Jungwook Choi, Daniel Brand, Chia-Yu Chen, and Kailash Gopalakrishnan. 2018. Training deep neural networks with 8-bit floating point numbers. In NeurIPS. 7686\u20137695."},{"key":"e_1_3_3_122_2","first-page":"945","volume-title":"NSDI","author":"Weng Qizhen","year":"2022","unstructured":"Qizhen Weng, Wencong Xiao, Yinghao Yu, Wei Wang, Cheng Wang, Jian He, Yong Li, Liping Zhang, Wei Lin, and Yu Ding. 2022. MLaaS in the wild: Workload analysis and scheduling in large-scale heterogeneous GPU clusters. In NSDI. 945\u2013960."},{"key":"e_1_3_3_123_2","unstructured":"Yonghui Wu Mike Schuster Zhifeng Chen Quoc V. Le Mohammad Norouzi Wolfgang Macherey Maxim Krikun Yuan Cao Qin Gao Klaus Macherey et\u00a0al. 2016. Google\u2019s neural machine translation system: Bridging the gap between human and machine translation. CoRR abs\/1609.08144 (2016). Retrieved from https:\/\/arxiv.org\/abs\/1609.08144"},{"key":"e_1_3_3_124_2","first-page":"533","volume-title":"OSDI","author":"Xiao Wencong","year":"2020","unstructured":"Wencong Xiao, Shiru Ren, Yong Li, Yang Zhang, Pengyang Hou, Zhi Li, Yihui Feng, Wei Lin, and Yangqing Jia. 2020. AntMan: Dynamic scaling on GPU clusters for deep learning. In OSDI. 533\u2013548."},{"key":"e_1_3_3_125_2","first-page":"225","volume-title":"PACT","author":"Xu Qiumin","year":"2014","unstructured":"Qiumin Xu and Murali Annavaram. 2014. PATS: Pattern aware scheduling and power gating for GPGPUs. In PACT. 225\u2013236."},{"key":"e_1_3_3_126_2","first-page":"479","volume-title":"HPCA","author":"Yeh Tsung Tai","year":"2021","unstructured":"Tsung Tai Yeh, Matthew D. Sinclair, Bradford M. Beckmann, and Timothy G. Rogers. 2021. Deadline-aware offloading for high-throughput accelerators. In HPCA. 479\u2013492."},{"key":"e_1_3_3_127_2","first-page":"15","volume-title":"ICS","author":"Yu Yulong","year":"2015","unstructured":"Yulong Yu, Weijun Xiao, Xubin He, He Guo, Yuxin Wang, and Xin Chen. 2015. A stall-aware warp scheduling for dynamically optimizing thread-level parallelism in GPGPUs. In ICS. 15\u201324."},{"key":"e_1_3_3_128_2","first-page":"131","volume-title":"IISWC","author":"Zadeh Ali Hadi","year":"2019","unstructured":"Ali Hadi Zadeh, Zissis Poulos, and Andreas Moshovos. 2019. Deep learning language modeling workloads: Where time goes on graphics processors. In IISWC. 131\u2013142."},{"key":"e_1_3_3_129_2","first-page":"8","volume-title":"HotCloud","author":"Zhang Jeff","year":"2020","unstructured":"Jeff Zhang, Sameh Elnikety, Shuayb Zarar, Atul Gupta, and Siddharth Garg. 2020. Model-switching: Dealing with fluctuating workloads in machine-learning-as-a-service systems. In HotCloud. 8 pages."},{"key":"e_1_3_3_130_2","first-page":"951","volume-title":"USENIX ATC","author":"Zhang Minjia","year":"2018","unstructured":"Minjia Zhang, Samyam Rajbhandari, Wenhan Wang, and Yuxiong He. 2018. DeepCPU: Serving RNN-based deep learning models 10x faster. In USENIX ATC. 951\u2013965."},{"key":"e_1_3_3_131_2","first-page":"15","volume-title":"ICLR","author":"Zhu Feiwen","year":"2018","unstructured":"Feiwen Zhu, Jeff Pool, Michael Andersch, Jeremy Appleyard, and Fung Xie. 2018. Sparse persistent RNNs: Squeezing large recurrent networks on-chip. In ICLR. 15 pages."}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3730584","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,1]],"date-time":"2025-07-01T08:31:48Z","timestamp":1751358708000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3730584"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,30]]},"references-count":130,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,6,30]]}},"alternative-id":["10.1145\/3730584"],"URL":"https:\/\/doi.org\/10.1145\/3730584","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"value":"1544-3566","type":"print"},{"value":"1544-3973","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,30]]},"assertion":[{"value":"2024-09-03","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-17","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}