{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T01:19:28Z","timestamp":1789435168039,"version":"build-2803163510"},"reference-count":20,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2025,8]]},"abstract":"<jats:p>Large language models (LLMs) are powering a new wave of language-based applications, including database applications, leading to new techniques and systems for dealing with the enormous compute and memory needs of LLMs, coupled with advances in computing hardware. In this tutorial, we review how these techniques lower inference costs by managing uncertain request lifecycles, exploiting specialized hardware, and scaling over distributed inference devices and machines. We present these techniques from the database perspective of request processing, model execution and optimization, and memory management. Following these discussion, we review how inference systems combine these techniques in diverse architectures to achieve application or performance objectives.<\/jats:p>","DOI":"10.14778\/3750601.3750703","type":"journal-article","created":{"date-parts":[[2025,9,16]],"date-time":"2025-09-16T13:37:51Z","timestamp":1758029871000},"page":"5504-5507","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Database Perspective on LLM Inference Systems"],"prefix":"10.14778","volume":"18","author":[{"given":"James","family":"Pan","sequence":"first","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guoliang","family":"Li","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,9,16]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"OSDI'24","author":"Agrawal Amey","year":"2024","unstructured":"Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gulavani, Alexey Tumanov, and Ramachandran Ramjee. 2024. Taming throughput-latency tradeoff in LLM inference with Sarathi-Serve. In OSDI'24. 117\u2013134."},{"key":"e_1_2_1_2_1","volume-title":"EMNLP'23","author":"Ainslie Joshua","year":"2023","unstructured":"Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebron, and Sumit Sanghai. 2023. GQA: Training generalized multi-query transformer models from multi-head checkpoints. In EMNLP'23. 4895\u20134901."},{"key":"e_1_2_1_3_1","volume-title":"AAAI'24 38","author":"Besta Maciej","year":"2024","unstructured":"Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. 2024. Graph of thoughts: Solving elaborate problems with large language models. AAAI'24 38, 16 (2024), 17682\u201317690."},{"key":"e_1_2_1_4_1","unstructured":"Zhuoming Chen Ranajoy Sadhukhan Zihao Ye Yang Zhou Jianyu Zhang Niklas Nolte Yuandong Tian Matthijs Douze Leon Bottou Zhihao Jia and Beidi Chen. 2024. MagicPIG: LSH sampling for efficient LLM generation. arXiv:2410.16179 [cs.CL] https:\/\/arxiv.org\/abs\/2410.16179"},{"key":"e_1_2_1_5_1","doi-asserted-by":"crossref","unstructured":"Krishna Teja Chitty-Venkata Siddhisanket Raskar Bharat Kale Farah Ferdaus Aditya Tanikanti Ken Raffenetti Valerie Taylor Murali Emani and Venkatram Vishwanath. 2024. LLM-Inference-Bench: Inference benchmarking of large language models on AI accelerators. arXiv:2411.00136 [cs.LG]","DOI":"10.1109\/SCW63240.2024.00178"},{"key":"e_1_2_1_6_1","unstructured":"Tri Dao Daniel Y. Fu Stefano Ermon Atri Rudra and Christopher R\u00e9. 2022. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. In NeurIPS'22."},{"key":"e_1_2_1_7_1","doi-asserted-by":"crossref","unstructured":"Alex Graves. 2012. Sequence transduction with recurrent neural networks. arXiv:1211.3711 [cs.NE] https:\/\/arxiv.org\/abs\/1211.3711","DOI":"10.1007\/978-3-642-24797-2_3"},{"key":"e_1_2_1_8_1","volume-title":"EPIC: Efficient position-independent context caching for serving large language models. arXiv:2410.15332 [cs.LG] https:\/\/arxiv.org\/abs\/2410.15332","author":"Hu Junhao","year":"2025","unstructured":"Junhao Hu, Wenrui Huang, Haoyi Wang, Weidong Wang, Tiancheng Hu, Qin Zhang, Hao Feng, Xusheng Chen, Yizhou Shan, and Tao Xie. 2025. EPIC: Efficient position-independent context caching for serving large language models. arXiv:2410.15332 [cs.LG] https:\/\/arxiv.org\/abs\/2410.15332"},{"key":"e_1_2_1_9_1","unstructured":"Junhao Hu Jiang Xu Zhixia Liu Yulong He Yuetao Chen Hao Xu Jiang Liu Baoquan Zhang Shining Wan Gengyuan Dan Zhiyu Dong Zhihao Ren Jie Meng Chao He Changhong Liu Tao Xie Dayun Lin Qin Zhang Yue Yu Hao Feng Xusheng Chen and Yizhou Shan. 2025. DeepFlow: Serverless large language model serving at scale. arXiv:2501.14417 [cs.DC]"},{"key":"e_1_2_1_10_1","unstructured":"Kyoungmin Kim and Anastasia Ailamaki. 2024. Trustworthy and efficient LLMs meet databases. arXiv:2412.18022 [cs.DB] https:\/\/arxiv.org\/abs\/2412.18022"},{"key":"e_1_2_1_11_1","unstructured":"Ferdi Kossmann Bruce Fontaine Daya Khudia Michael Cafarella and Samuel Madden. 2025. Is the GPU half-empty or half-full? Practical scheduling techniques for LLMs. arXiv:2410.17840 [cs.LG] https:\/\/arxiv.org\/abs\/2410.17840"},{"key":"e_1_2_1_12_1","volume-title":"SOSP'23","author":"Kwon Woosuk","year":"2023","unstructured":"Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with PagedAttention. In SOSP'23. 611\u2013626."},{"key":"e_1_2_1_13_1","volume-title":"ICML'23","author":"Leviathan Yaniv","year":"2023","unstructured":"Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via speculative decoding. In ICML'23."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.14778\/3685800.3685838"},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1145\/3714983.3714987","article-title":"AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration","volume":"28","author":"Lin Ji","year":"2025","unstructured":"Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Guangxuan Xiao, and Song Han. 2025. AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration. GetMobile 28, 4 (2025), 12\u201317.","journal-title":"GetMobile"},{"key":"e_1_2_1_16_1","volume-title":"Mooncake: A KVCache-centric disaggregated architecture for LLM serving. arXiv:2407.00079 [cs.DC]","author":"Qin Ruoyu","year":"2024","unstructured":"Ruoyu Qin, Zheming Li, Weiran He, Mingxing Zhang, Yongwei Wu, Weimin Zheng, and Xinran Xu. 2024. Mooncake: A KVCache-centric disaggregated architecture for LLM serving. arXiv:2407.00079 [cs.DC]"},{"key":"e_1_2_1_17_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS'17. 6000\u20136010."},{"key":"e_1_2_1_18_1","volume-title":"ICLR'23","author":"Wang Xuezhi","year":"2023","unstructured":"Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In ICLR'23."},{"key":"e_1_2_1_19_1","volume-title":"OSDI'22","author":"Yu Gyeong-In","year":"2022","unstructured":"Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. 2022. Orca: A distributed serving system for transformer-based generative models. In OSDI'22. 521\u2013538."},{"key":"e_1_2_1_20_1","volume-title":"Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng.","author":"Zheng Lianmin","year":"2024","unstructured":"Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. 2024. SGLang: Efficient execution of structured language model programs. arXiv:2312.07104 [cs.AI] https:\/\/arxiv.org\/abs\/2312.07104"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3750601.3750703","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,16]],"date-time":"2025-09-16T13:38:38Z","timestamp":1758029918000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3750601.3750703"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8]]},"references-count":20,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2025,8]]}},"alternative-id":["10.14778\/3750601.3750703"],"URL":"https:\/\/doi.org\/10.14778\/3750601.3750703","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2025,8]]},"assertion":[{"value":"2025-09-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}