{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T20:30:51Z","timestamp":1792787451261,"version":"4.5.0"},"reference-count":37,"publisher":"Association for Computing Machinery (ACM)","issue":"1","funder":[{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"crossref","award":["62472377, 62502434, 62402422"],"award-info":[{"award-number":["62472377, 62502434, 62402422"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Ningbo Yongjiang Talent Introduction Programme","award":["2022A-237-G"],"award-info":[{"award-number":["2022A-237-G"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2026,4,2]]},"abstract":"<jats:p>Multi-modal analytical processing has the potential to transform applications in e-commerce, healthcare, entertainment, and beyond. However, real-world adoption remains elusive due to the limited ability of traditional relational query operators to capture query semantics. The emergence of foundation models, particularly the large language models (LLMs), opens up new opportunities to develop flexible, semantic-aware data analytics systems that transcend the relational paradigm.<\/jats:p>\n                  <jats:p>We present Nirvana, a multi-modal data analytics framework that incorporates programmable semantic operators while leveraging both logical and physical query optimization strategies, tailored for LLM-driven semantic query processing. Nirvana addresses two key challenges. First, it features an agentic logical optimizer that uses natural language-specified transformation rules and random-walk-based search to explore vast spaces of semantically equivalent query plans --- far beyond the capabilities of conventional optimizers. Second, it introduces a cost-aware physical optimizer that selects the most effective LLM backend for each operator using a novel improvement-score metric. To further enhance efficiency, Nirvana incorporates computation reuse and evaluation pushdown techniques guided by model capability hypotheses. Experimental evaluations on three real-world benchmarks demonstrate that Nirvana is able to reduce end-to-end runtime by 10%\u201385% and reduces system processing costs by 76% on average, outperforming state-of-the-art systems at both efficiency and scalability.<\/jats:p>","DOI":"10.1145\/3786628","type":"journal-article","created":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T17:54:13Z","timestamp":1775584453000},"page":"1-25","source":"Crossref","is-referenced-by-count":1,"title":["Beyond Relational: Semantic-Aware Multi-Modal Analytics with LLM-Native Query Optimization"],"prefix":"10.1145","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-0531-5004","authenticated-orcid":false,"given":"Junhao","family":"Zhu","sequence":"first","affiliation":[{"name":"Zhejiang University, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5685-7017","authenticated-orcid":false,"given":"Lu","family":"Chen","sequence":"additional","affiliation":[{"name":"Zhejiang University, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8082-7398","authenticated-orcid":false,"given":"Xiangyu","family":"Ke","sequence":"additional","affiliation":[{"name":"Zhejiang University, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-2034-5501","authenticated-orcid":false,"given":"Ziquan","family":"Fang","sequence":"additional","affiliation":[{"name":"Zhejiang University, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5424-6442","authenticated-orcid":false,"given":"Tianyi","family":"Li","sequence":"additional","affiliation":[{"name":"Aalborg University, Aalborg, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3816-8450","authenticated-orcid":false,"given":"Yunjun","family":"Gao","sequence":"additional","affiliation":[{"name":"Zhejiang University, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9697-7670","authenticated-orcid":false,"given":"Chrisitian S.","family":"Jensen","sequence":"additional","affiliation":[{"name":"Aalborg University, Aalborg, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,4,7]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Amazon. 2025. S3: Cloud Object Storage. https:\/\/aws.amazon.com\/s3\/."},{"key":"e_1_2_1_2_1","volume-title":"Bohou Li, Mark Lindblad, Henry Lindeman, Alex Meyer, Parth Parmar, Tanvi Ranade, Mehul A. Shah, Benjamin Sowell, Dan Tecuci, Vinayak Thapliyal, and Matt Welsh.","author":"Anderson Eric","year":"2024","unstructured":"Eric Anderson, Jonathan Fritz, Austin Lee, Bohou Li, Mark Lindblad, Henry Lindeman, Alex Meyer, Parth Parmar, Tanvi Ranade, Mehul A. Shah, Benjamin Sowell, Dan Tecuci, Vinayak Thapliyal, and Matt Welsh. 2024. The Design of an LLM-powered Unstructured Analytics System. CoRR, Vol. abs\/2409.00847 (2024)."},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Michael Armbrust Reynold S. Xin Cheng Lian Yin Huai Davies Liu Joseph K. Bradley Xiangrui Meng Tomer Kaftan Michael J. Franklin Ali Ghodsi and Matei Zaharia. 2015. Spark SQL: Relational Data Processing in Spark. In SIGMOD Timos K. Sellis Susan B. Davidson and Zachary G. Ives (Eds.). 1383-1394.","DOI":"10.1145\/2723372.2742797"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.14778\/3749646.3749647"},{"key":"e_1_2_1_5_1","unstructured":"Si-An Chen Lesly Miculicich Julian Eisenschlos Zifeng Wang Zilong Wang Yanfei Chen Yasuhisa Fujii Hsuan-Tien Lin Chen-Yu Lee and Tomas Pfister. 2024. TableRAG: Million-Token Table Understanding with Language Models. In NeurIPS. http:\/\/papers.nips.cc\/paper_files\/paper\/2024\/hash\/88dd7aa6979e352fda7c4952ca8eac59-Abstract-Conference.html"},{"key":"e_1_2_1_6_1","unstructured":"Zhoujun Cheng Tianbao Xie Peng Shi Chengzu Li Rahul Nadkarni Yushi Hu Caiming Xiong Dragomir Radev Mari Ostendorf Luke Zettlemoyer Noah A. Smith and Tao Yu. 2023. Binding Language Models in Symbolic Languages. In ICLR. https:\/\/openreview.net\/forum?id=lH1PV42cbF"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626768"},{"key":"e_1_2_1_8_1","first-page":"19","article-title":"The Cascades Framework for Query Optimization","volume":"18","author":"Graefe Goetz","year":"1995","unstructured":"Goetz Graefe. 1995. The Cascades Framework for Query Optimization. IEEE Data Eng. Bull., Vol. 18, 3 (1995), 19-29.","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_2_1_9_1","volume-title":"McKenna","author":"Graefe Goetz","year":"1993","unstructured":"Goetz Graefe and William J. McKenna. 1993. The Volcano Optimizer Generator: Extensibility and Efficient Search. In ICDE. 209-218."},{"key":"e_1_2_1_10_1","first-page":"855","article-title":"node2vec","author":"Grover Aditya","year":"2016","unstructured":"Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable Feature Learning for Networks. In SIGKDD. 855-864.","journal-title":"Scalable Feature Learning for Networks. In SIGKDD."},{"key":"e_1_2_1_11_1","unstructured":"Edward J. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In ICLR. https:\/\/openreview.net\/forum?id=nZeVKeeFYf9"},{"key":"e_1_2_1_12_1","unstructured":"Shengran Hu Cong Lu and Jeff Clune. 2025. Automated Design of Agentic Systems. In ICLR. https:\/\/openreview.net\/forum?id=t9U3LW7JVX"},{"key":"e_1_2_1_13_1","volume-title":"SMART: Automatically Scaling Down Language Models with Accuracy Guarantees for Reduced Processing Fees. CoRR","author":"Jo Saehan","year":"2024","unstructured":"Saehan Jo and Immanuel Trummer. 2024a. SMART: Automatically Scaling Down Language Models with Accuracy Guarantees for Reduced Processing Fees. CoRR, Vol. abs\/2403.13835 (2024)."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3654989"},{"key":"e_1_2_1_15_1","first-page":"611","article-title":"Efficient Memory Management for Large Language Model Serving with PagedAttention","author":"Kwon Woosuk","year":"2023","unstructured":"Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In SOSP. 611-626.","journal-title":"SOSP."},{"key":"e_1_2_1_16_1","volume-title":"Towards Accurate and Efficient Document Analytics with Large Language Models. CoRR","author":"Lin Yiming","year":"2024","unstructured":"Yiming Lin, Madelon Hulsebos, Ruiying Ma, Shreya Shankar, Sepanta Zeighami, Aditya G. Parameswaran, and Eugene Wu. 2024. Towards Accurate and Efficient Document Analytics with Large Language Models. CoRR, Vol. abs\/2405.04674 (2024)."},{"key":"e_1_2_1_17_1","volume-title":"Zui Chen, Michael J. Franklin, Tim Kraska, Samuel Madden, and Gerardo Vitagliano.","author":"Liu Chunwei","year":"2025","unstructured":"Chunwei Liu, Matthew Russo, Michael J. Cafarella, Lei Cao, Peter Baile Chen, Zui Chen, Michael J. Franklin, Tim Kraska, Samuel Madden, and Gerardo Vitagliano. 2025b. Palimpzest: Optimizing AI-Powered Analytics with Declarative Query Processing. In CIDR."},{"key":"e_1_2_1_18_1","volume-title":"Liana Patel, Shiyi Cao, Xiangxi Mo, Ion Stoica, Joseph E. Gonzalez, and Matei Zaharia.","author":"Liu Shu","year":"2025","unstructured":"Shu Liu, Asim Biswal, Amog Kamsetty, Audrey Cheng, Luis Gaspar Schroeder, Liana Patel, Shiyi Cao, Xiangxi Mo, Ion Stoica, Joseph E. Gonzalez, and Matei Zaharia. 2025a. Optimizing LLM Queries in Relational Data Analytics Workloads. In MLsys."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.14778\/3685800.3685916"},{"key":"e_1_2_1_20_1","unstructured":"OpenAI. 2025. Pricing policy. https:\/\/openai.com\/api\/pricing\/."},{"key":"e_1_2_1_21_1","volume-title":"Semantic Operators: A Declarative Model for Rich, AI-based Analytics Over Text Data. CoRR","author":"Patel Liana","year":"2024","unstructured":"Liana Patel, Siddharth Jha, Parth Asawa, Melissa Pan, Carlos Guestrin, and Matei Zaharia. 2024. Semantic Operators: A Declarative Model for Rich, AI-based Analytics Over Text Data. CoRR, Vol. abs\/2407.11418 (2024)."},{"key":"e_1_2_1_22_1","first-page":"3980","article-title":"Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks","author":"Reimers Nils","year":"2019","unstructured":"Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In EMNLP-IJCNLP. 3980-3990.","journal-title":"EMNLP-IJCNLP."},{"key":"e_1_2_1_23_1","volume-title":"Abacus: A Cost-Based Optimizer for Semantic Operator Systems. CoRR","author":"Russo Matthew","year":"2025","unstructured":"Matthew Russo, Sivaprasad Sudhir, Gerardo Vitagliano, Chunwei Liu, Tim Kraska, Samuel Madden, and Michael Cafarella. 2025. Abacus: A Cost-Based Optimizer for Semantic Operator Systems. CoRR, Vol. abs\/2505.14661 (2025)."},{"key":"e_1_2_1_24_1","volume-title":"DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing. CoRR","author":"Shankar Shreya","year":"2024","unstructured":"Shreya Shankar, Aditya G. Parameswaran, and Eugene Wu. 2024. DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing. CoRR, Vol. abs\/2410.12189 (2024)."},{"key":"e_1_2_1_25_1","unstructured":"Yongliang Shen Kaitao Song Xu Tan Dongsheng Li Weiming Lu and Yueting Zhuang. 2023. HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face. In NeurIPS. http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/77c33e6a367922d003ff102ffb92b658-Abstract-Conference.html"},{"key":"e_1_2_1_26_1","unstructured":"MindsDB Team. 2025. MindsDB: AI's query engine. https:\/\/mindsdb.com\/."},{"key":"e_1_2_1_27_1","unstructured":"TPC-H. 2025. TPC-H V3 Top Performance Results 30-Jun-2025 at 9:01 AM. https:\/\/www.tpc.org\/tpch\/results\/tpch_perf_results5.asp?resulttype=all&version=3."},{"key":"e_1_2_1_28_1","volume-title":"CAESURA: Language Models as Multi-Modal Query Planners. In CIDR. https:\/\/www.cidrdb.org\/cidr2024\/papers\/p14-urban.pdf","author":"Urban Matthias","year":"2024","unstructured":"Matthias Urban and Carsten Binnig. 2024. CAESURA: Language Models as Multi-Modal Query Planners. In CIDR. https:\/\/www.cidrdb.org\/cidr2024\/papers\/p14-urban.pdf"},{"key":"e_1_2_1_29_1","first-page":"57","article-title":"iDataLake: An LLM-Powered Analytics System on Data Lakes","volume":"49","author":"Wang Jiayi","year":"2025","unstructured":"Jiayi Wang, Guoliang Li, and Jianhua Feng. 2025. iDataLake: An LLM-Powered Analytics System on Data Lakes. IEEE Data Eng. Bull., Vol. 49, 1 (2025), 57-69.","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_2_1_30_1","volume-title":"No more optimization rules: LLM-enabled policy-based multi-modal query optimizer. CoRR","author":"Wang Yifan","year":"2024","unstructured":"Yifan Wang, Haodi Ma, and Daisy Zhe Wang. 2024. No more optimization rules: LLM-enabled policy-based multi-modal query optimizer. CoRR, Vol. abs\/2403.13597 (2024)."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3725271"},{"key":"e_1_2_1_32_1","unstructured":"Jiayi Zhang Jinyu Xiang Zhaoyang Yu Fengwei Teng Xionghui Chen Jiaqi Chen Mingchen Zhuge Xin Cheng Sirui Hong Jinlin Wang Bingnan Zheng Bang Liu Yuyu Luo and Chenglin Wu. 2025. AFlow: Automating Agentic Workflow Generation. In ICLR. https:\/\/openreview.net\/forum?id=z5uVAKwmjf"},{"key":"e_1_2_1_33_1","unstructured":"Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu Yonghao Zhuang Zi Lin Zhuohan Li Dacheng Li Eric P. Xing Hao Zhang Joseph E. Gonzalez and Ion Stoica. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In NeurIPS. http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/91f18a1287b398d378ef22505bf41832-Abstract-Datasets_and_Benchmarks.html"},{"key":"e_1_2_1_34_1","volume-title":"Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark W. Barrett, and Ying Sheng.","author":"Zheng Lianmin","year":"2024","unstructured":"Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark W. Barrett, and Ying Sheng. 2024b. SGLang: Efficient Execution of Structured Language Model Programs. In NeurIPS. http:\/\/papers.nips.cc\/paper_files\/paper\/2024\/hash\/724be4472168f31ba1c9ac630f15dec8-Abstract-Conference.html"},{"key":"e_1_2_1_35_1","first-page":"9102","article-title":"Multimodal Table Understanding","author":"Zheng Mingyu","year":"2024","unstructured":"Mingyu Zheng, Xinwei Feng, Qingyi Si, Qiaoqiao She, Zheng Lin, Wenbin Jiang, and Weiping Wang. 2024a. Multimodal Table Understanding. In ACL. 9102-9124.","journal-title":"ACL."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2511.19830"},{"key":"e_1_2_1_37_1","unstructured":"Mingchen Zhuge Wenyi Wang Louis Kirsch Francesco Faccio Dmitrii Khizbullin and J\u00fcrgen Schmidhuber. 2024. GPTSwarm: Language Agents as Optimizable Graphs. In ICML. https:\/\/openreview.net\/forum?id=uTC9AFXIhg"}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3786628","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T19:58:05Z","timestamp":1775591885000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3786628"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,2]]},"references-count":37,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,4,2]]}},"alternative-id":["10.1145\/3786628"],"URL":"https:\/\/doi.org\/10.1145\/3786628","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,2]]}}}