{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,15]],"date-time":"2026-03-15T15:30:37Z","timestamp":1773588637466,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":47,"publisher":"ACM","funder":[{"name":"NSF (National Science Foundation)","award":["CCF-2450085"],"award-info":[{"award-number":["CCF-2450085"]}]},{"name":"NSF (National Science Foundation)","award":["CNS-2106184"],"award-info":[{"award-number":["CNS-2106184"]}]},{"name":"NSF (National Science Foundation)","award":["CNS-2214272"],"award-info":[{"award-number":["CNS-2214272"]}]},{"name":"NSF (National Science Foundation)","award":["CNS-2106751"],"award-info":[{"award-number":["CNS-2106751"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,3,22]]},"DOI":"10.1145\/3779212.3790233","type":"proceedings-article","created":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T13:55:26Z","timestamp":1773150926000},"page":"1982-1997","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["TetriServe: Efficiently Serving Mixed DiT Workloads"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-5261-6147","authenticated-orcid":false,"given":"Runyu","family":"Lu","sequence":"first","affiliation":[{"name":"University of Michigan, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-2367-2827","authenticated-orcid":false,"given":"Shiqi","family":"He","sequence":"additional","affiliation":[{"name":"University of Michigan, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-1630-3194","authenticated-orcid":false,"given":"Wenxuan","family":"Tan","sequence":"additional","affiliation":[{"name":"University of Wisconsin-Madison, Madison, Wisconsin, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2037-2496","authenticated-orcid":false,"given":"Shenggui","family":"Li","sequence":"additional","affiliation":[{"name":"Nanyang Technological University, Singapore, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6826-8108","authenticated-orcid":false,"given":"Ruofan","family":"Wu","sequence":"additional","affiliation":[{"name":"University of Michigan, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-9520-5218","authenticated-orcid":false,"given":"Jeff J.","family":"Ma","sequence":"additional","affiliation":[{"name":"University of Michigan, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-8326-8124","authenticated-orcid":false,"given":"Ang","family":"Chen","sequence":"additional","affiliation":[{"name":"University of Michigan, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0884-6740","authenticated-orcid":false,"given":"Mosharaf","family":"Chowdhury","sequence":"additional","affiliation":[{"name":"University of Michigan, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,3,22]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3555765"},{"key":"e_1_3_2_1_2_1","unstructured":"Shubham Agarwal Subrata Mitra Sarthak Chakraborty Srikrishna Karanam Koyel Mukherjee and Shiv Kumar Saini. 2024. Approximate Caching for Efficiently Serving Text-to-Image Diffusion Models. In NSDI."},{"key":"e_1_3_2_1_3_1","unstructured":"Stability AI. 2024a. Stable Diffusion 3 Medium. https:\/\/huggingface.co\/stabilityai\/stable-diffusion-3-medium."},{"key":"e_1_3_2_1_4_1","unstructured":"Stability AI. 2024b. Stable Diffusion 3.5 Large. https:\/\/huggingface.co\/stabilityai\/stable-diffusion-3.5-large."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"crossref","unstructured":"Amotz Bar-Noy Sudipto Guha Joseph Naor and Baruch Schieber. 1999. Approximating the Throughput of Multiple Machines under Real-Time Scheduling. In STOC.","DOI":"10.1145\/301250.301420"},{"key":"e_1_3_2_1_6_1","unstructured":"Hugo Barbalho Patricia Kovaleski Beibin Li Luke Marshall Marco Molinaro Abhisek Pan Eli Cortez Matheus Leao Harsh Patwari Zuzu Tang et al. 2023. Virtual Machine Allocation with Lifetime Predictions. In MLSys."},{"key":"e_1_3_2_1_7_1","unstructured":"Tim Brooks Bill Peebles Connor Holmes Will DePue Yufei Guo Li Jing David Schnurr Joe Taylor Troy Luhman Eric Luhman et al. 2024. Video Generation Models as World Simulators. OpenAI Blog (2024)."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"crossref","unstructured":"Zigeng Chen Xinyin Ma Gongfan Fang Zhenxiong Tan and Xinchao Wang. 2024. AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising. NeurIPS.","DOI":"10.52202\/079017-3015"},{"key":"e_1_3_2_1_9_1","unstructured":"Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly Jakob Uszkoreit and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR."},{"key":"e_1_3_2_1_10_1","unstructured":"Kuntai Du Bowen Wang Chen Zhang Yiming Cheng Qing Lan Hejian Sang Yihua Cheng Jiayi Yao Xiaoxuan Liu Yifan Qiao Ion Stoica and Junchen Jiang. 2025. PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications. In SOSP."},{"key":"e_1_3_2_1_11_1","unstructured":"Jiangfei Duan Runyu Lu Haojie Duanmu Xiuhong Li Xingcheng Zhang Dahua Lin Ion Stoica and Hao Zhang. 2024. MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving. In ICML."},{"key":"e_1_3_2_1_12_1","volume-title":"xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism. arXiv preprint arXiv:2411.01738","author":"Fang Jiarui","year":"2024","unstructured":"Jiarui Fang, Jinzhe Pan, Xibo Sun, Aoyu Li, and Jiannan Wang. 2024. xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism. arXiv preprint arXiv:2411.01738 (2024)."},{"key":"e_1_3_2_1_13_1","unstructured":"Flux.1 AI. 2025. Flux.1 AI Image Generator. https:\/\/flux1.ai\/create"},{"key":"e_1_3_2_1_14_1","volume-title":"Johnson","author":"Garey Michael R","year":"1977","unstructured":"Michael R Garey and David S. Johnson. 1977. Two-Processor Scheduling with Start-Times and Deadlines. SIAM journal on Computing (1977)."},{"key":"e_1_3_2_1_15_1","volume-title":"Hongqiang Harry Liu, and Chuanxiong Guo","author":"Gu Juncheng","year":"2019","unstructured":"Juncheng Gu, Mosharaf Chowdhury, Kang G. Shin, Yibo Zhu, Myeongjae Jeon, Junjie Qian, Hongqiang Harry Liu, and Chuanxiong Guo. 2019. Tiresias: A GPU Cluster Manager for Distributed Deep Learning. In NSDI."},{"key":"e_1_3_2_1_16_1","unstructured":"Jonathan Ho Ajay Jain and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In NeurIPS."},{"key":"e_1_3_2_1_17_1","volume-title":"DDiT: Dynamic Resource Allocation for Diffusion Transformer Model Serving. arXiv preprint arXiv:2506.13497","author":"Huang Heyang","year":"2025","unstructured":"Heyang Huang, Cunchen Hu, Jiaqi Zhu, Ziyuan Gao, Liangliang Xu, Yizhou Shan, Yungang Bao, Sun Ninghui, Tianwei Zhang, and Sa Wang. 2025. DDiT: Dynamic Resource Allocation for Diffusion Transformer Model Serving. arXiv preprint arXiv:2506.13497 (2025)."},{"key":"e_1_3_2_1_18_1","volume-title":"Samyam Rajbhandari, and Yuxiong He.","author":"Jacobs Sam Ade","year":"2023","unstructured":"Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. 2023. DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models. arXiv preprint arXiv:2309.14509 (2023)."},{"key":"e_1_3_2_1_19_1","unstructured":"Alind Khare Dhruv Garg Sukrit Kalra Snigdha Grandhi Ion Stoica and Alexey Tumanov. 2025. SuperServe: Fine-Grained Inference Serving for Unpredictable Workloads. In NSDI."},{"key":"e_1_3_2_1_20_1","volume-title":"Joseph Gonzalez, Hao Zhang, and Ion Stoica.","author":"Kwon Woosuk","year":"2023","unstructured":"Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In SOSP."},{"key":"e_1_3_2_1_21_1","unstructured":"Black Forest Labs. 2024. FLUX.1-dev: Text-to-Image Generation Model."},{"key":"e_1_3_2_1_22_1","unstructured":"Tan N. Le Xiao Sun Mosharaf Chowdhury and Zhenhua Liu. 2020. AlloX: Compute Allocation in Hybrid Clusters. In EuroSys."},{"key":"e_1_3_2_1_23_1","volume-title":"Sequence Parallelism: Long Sequence Training from System Perspective. In ACL.","author":"Li Shenggui","year":"2023","unstructured":"Shenggui Li, Fuzhao Xue, Chaitanya Baranwal, Yongbin Li, and Yang You. 2023. Sequence Parallelism: Long Sequence Training from System Perspective. In ACL."},{"key":"e_1_3_2_1_24_1","volume-title":"Katz: Efficient Workflow Serving for Diffusion Models with Many Adapters. In ATC.","author":"Li Suyi","year":"2025","unstructured":"Suyi Li, Lingyun Yang, Xiaoxiao Jiang, Hanfeng Lu, Dakai An, Zhipeng Di, Weiyi Lu, Jiawei Chen, Kan Liu, Yinghao Yu, Tao Lan, Guodong Yang, Lin Qu, Liping Zhang, and Wei Wang. 2025. Katz: Efficient Workflow Serving for Diffusion Models with Many Adapters. In ATC."},{"key":"e_1_3_2_1_25_1","volume-title":"Ring Attention with Blockwise Transformers for Near-Infinite Context. arXiv preprint arXiv:2310.01889","author":"Liu Hao","year":"2023","unstructured":"Hao Liu, Matei Zaharia, and Pieter Abbeel. 2023. Ring Attention with Blockwise Transformers for Near-Infinite Context. arXiv preprint arXiv:2310.01889 (2023)."},{"key":"e_1_3_2_1_26_1","volume-title":"Michael Bi Mi, and Xinchao Wang","author":"Ma Xinyin","year":"2024","unstructured":"Xinyin Ma, Gongfan Fang, Michael Bi Mi, and Xinchao Wang. 2024. Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching. NeurIPS."},{"key":"e_1_3_2_1_27_1","volume-title":"Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow. In ASPLOS.","author":"Mei Yixuan","year":"2025","unstructured":"Yixuan Mei, Yonghao Zhuang, Xupeng Miao, Juncheng Yang, Zhihao Jia, and Rashmi Vinayak. 2025. Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow. In ASPLOS."},{"key":"e_1_3_2_1_28_1","volume-title":"Alan Zhu, Lijie Yang, Xiaoxiang Shi, Chunan Shi, Zhuoming Chen, Daiyaan Arfeen, Reyna Abhyankar, and Zhihao Jia.","author":"Miao Xupeng","year":"2024","unstructured":"Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu, Lijie Yang, Xiaoxiang Shi, Chunan Shi, Zhuoming Chen, Daiyaan Arfeen, Reyna Abhyankar, and Zhihao Jia. 2024a. SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification. In ASPLOS."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"crossref","unstructured":"Xupeng Miao Chunan Shi Jiangfei Duan Xiaoli Xi Dahua Lin Bin Cui and Zhihao Jia. 2024b. SpotServe: Serving Generative Large Language Models on Preemptible Instances. In ASPLOS.","DOI":"10.1145\/3620665.3640411"},{"key":"e_1_3_2_1_30_1","unstructured":"Deepak Narayanan Keshav Santhanam Fiodar Kazhamiaka Amar Phanishayee and Matei Zaharia. 2020. Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads. In OSDI."},{"key":"e_1_3_2_1_31_1","unstructured":"NVIDIA. 2022. NVIDIA Collective Communication Library (NCCL) Documentation. https:\/\/docs.nvidia.com\/deeplearning\/nccl\/user-guide\/docs\/index.html."},{"key":"e_1_3_2_1_32_1","volume-title":"Combinatorial Optimization: Algorithms and Complexity","author":"Papadimitriou Christos H","year":"1998","unstructured":"Christos H Papadimitriou and Kenneth Steiglitz. 1998. Combinatorial Optimization: Algorithms and Complexity. Courier Corporation."},{"key":"e_1_3_2_1_33_1","volume-title":"Saeed Maleki, and Ricardo Bianchini.","author":"Patel Pratyush","year":"2024","unstructured":"Pratyush Patel, Esha Choukse, Chaojie Zhang, Aashaka Shah, \u00cd nigo Goiri, Saeed Maleki, and Ricardo Bianchini. 2024. Splitwise: Efficient Generative LLM Inference Using Phase Splitting. In ISCA."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"crossref","unstructured":"William Peebles and Saining Xie. 2023. Scalable Diffusion Models with Transformers. In ICCV.","DOI":"10.1109\/ICCV51070.2023.00387"},{"key":"e_1_3_2_1_35_1","volume-title":"Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al., 2021. Learning Transferable Visual Models From Natural Language Supervision. In ICML."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"crossref","unstructured":"Olaf Ronneberger Philipp Fischer and Thomas Brox. 2015. U-Net: Convolutional networks for biomedical image segmentation. In MICCAI.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_3_2_1_37_1","volume-title":"Deep unsupervised learning using nonequilibrium thermodynamics. arXiv preprint arXiv:1503.03585","author":"Sohl-Dickstein Jascha","year":"2015","unstructured":"Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015. Deep unsupervised learning using nonequilibrium thermodynamics. arXiv preprint arXiv:1503.03585 (2015)."},{"key":"e_1_3_2_1_38_1","unstructured":"Yang Song and Stefano Ermon. 2021. Score-Based Generative Modeling through Stochastic Differential Equations. In ICLR."},{"key":"e_1_3_2_1_39_1","unstructured":"Stability AI. 2024. Stability AI Platform API Reference. https:\/\/platform.stability.ai\/docs\/api-reference Accessed: 2024-11-26."},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"crossref","unstructured":"Desen Sun Zepeng Zhao and Yuke Wang. 2026. MixFusion: A Patch-Level Parallel Serving System for Mixed-Resolution Diffusion Models. In PPoPP.","DOI":"10.1145\/3774934.3786420"},{"key":"e_1_3_2_1_41_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is All You Need. In NeurIPS."},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"crossref","unstructured":"Zijie J Wang Evan Montoya David Munechika Haoyang Yang Benjamin Hoover and Duen Horng Chau. 2023. DiffusionDB: A Large-Scale Prompt Gallery Dataset for Text-to-Image Generative Models. In ACL.","DOI":"10.18653\/v1\/2023.acl-long.51"},{"key":"e_1_3_2_1_43_1","unstructured":"Bingyang Wu Shengyu Liu Yinmin Zhong Peng Sun Xuanzhe Liu and Xin Jin. 2024. LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism. In SOSP."},{"key":"e_1_3_2_1_44_1","unstructured":"Yuchen Xia Divyam Sharma Yichao Yuan Souvik Kundu and Nishil Talati. 2026. MoDM: Efficient Serving for Image Generation via Mixture-of-Diffusion Models. In ASPLOS."},{"key":"e_1_3_2_1_45_1","volume-title":"SHEPHERD: Serving DNNs in the Wild. In NSDI.","author":"Zhang Hong","year":"2023","unstructured":"Hong Zhang, Yupeng Tang, Anurag Khandelwal, and Ion Stoica. 2023. SHEPHERD: Serving DNNs in the Wild. In NSDI."},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"crossref","unstructured":"Yanli Zhao Andrew Gu Rohan Varma Liang Luo Chien-Chin Huang Min Xu Less Wright Hamid Shojanazeri Myle Ott Sam Shleifer et al. 2023. PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel. In VLDB.","DOI":"10.14778\/3611540.3611569"},{"key":"e_1_3_2_1_47_1","volume-title":"Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al.","author":"Zheng Lianmin","year":"2024","unstructured":"Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Livia Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al., 2024. SGLang: Efficient Execution of Structured Language Model Programs. In NeurIPS."}],"event":{"name":"ASPLOS '26: 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems","location":"Pittsburgh PA USA","sponsor":["SIGOPS ACM Special Interest Group on Operating Systems","SIGPLAN ACM Special Interest Group on Programming Languages","SIGARCH ACM Special Interest Group on Computer Architecture","SIGBED ACM Special Interest Group on Embedded Systems"]},"container-title":["Proceedings of the 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2"],"original-title":[],"deposited":{"date-parts":[[2026,3,15]],"date-time":"2026-03-15T14:00:08Z","timestamp":1773583208000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3779212.3790233"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,22]]},"references-count":47,"alternative-id":["10.1145\/3779212.3790233","10.1145\/3779212"],"URL":"https:\/\/doi.org\/10.1145\/3779212.3790233","relation":{},"subject":[],"published":{"date-parts":[[2026,3,22]]},"assertion":[{"value":"2026-03-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}