{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T19:07:27Z","timestamp":1779131247608,"version":"3.51.4"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2026,5,18]]},"abstract":"<jats:p>\n                    As vector databases become increasingly critical to supporting large language models (LLMs), efficient and effective vector data retrieval has become imperative.\n                    <jats:italic toggle=\"yes\">Filtered vector search<\/jats:italic>\n                    , which retrieves relevant vectors under scalar filter constraints, has recently attracted significant attention. However, existing benchmarks for evaluating filtered vector search have three major limitations. First, they rely on simple, low-dimensional datasets that fail to reflect real-world scenarios involving thousands of dimensions and millions of records. Second, they use random filter values, resulting in trivial query plans that do not adequately challenge modern vector database optimizers. Third, they lack a unified evaluation metric to comprehensively quantify end-to-end performance.\n                  <\/jats:p>\n                  <jats:p>\n                    In this work, we propose VecBench, a controllable benchmark for holistic evaluation of filtered vector search. Designing such a benchmark poses three key challenges: (1) generating high-dimensional and large-scale vector data while preserving the original similarity distribution, (2) constructing representative benchmark queries that capture diverse filtered search strategies, and (3) developing a unified metric that enables fair and comprehensive comparisons under both static and dynamic scenarios. VecBench addresses these challenges through\n                    <jats:italic toggle=\"yes\">controllable data generation, adjustable query synthesis,<\/jats:italic>\n                    and\n                    <jats:italic toggle=\"yes\">end-to-end holistic benchmarking.<\/jats:italic>\n                    First, it can generate massive vector data with flexible dimensionality and scale while preserving the original distribution with a provable error bound. Second, it supports fine-grained workload control by varying selectivity and filter correlation, enabling systematic stress testing of different filtered search strategies. Third, it introduces a unified evaluation framework covering six phases, including concurrent query processing and dynamic update scenarios. To demonstrate the effectiveness of VecBench and to shed light on the strengths and weaknesses of different filtered vector search methods, we conduct extensive experiments on ten representative approaches across four popular vector databases.\n                  <\/jats:p>","DOI":"10.1145\/3802125","type":"journal-article","created":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T18:19:16Z","timestamp":1779128356000},"page":"1-27","source":"Crossref","is-referenced-by-count":0,"title":["VecBench: A Controllable Benchmark for Filtered Vector Search: [Experiments &amp; Analysis]"],"prefix":"10.1145","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-8816-034X","authenticated-orcid":false,"given":"Xiang","family":"Zhang","sequence":"first","affiliation":[{"name":"Renmin University of China, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8924-7629","authenticated-orcid":false,"given":"Chao","family":"Zhang","sequence":"additional","affiliation":[{"name":"Renmin University of China, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4729-9903","authenticated-orcid":false,"given":"Ju","family":"Fan","sequence":"additional","affiliation":[{"name":"Renmin University of China, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1398-0621","authenticated-orcid":false,"given":"Guoliang","family":"Li","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5757-9135","authenticated-orcid":false,"given":"Xiaoyong","family":"Du","sequence":"additional","affiliation":[{"name":"Renmin University of China, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,5,18]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Quicker adc: Unlocking the hidden potential of product quantization with simd","author":"Andr\u00e9 Fabien","year":"2019","unstructured":"Fabien Andr\u00e9, Anne-Marie Kermarrec, and Nicolas Le Scouarnec. 2019. Quicker adc: Unlocking the hidden potential of product quantization with simd. IEEE transactions on pattern analysis and machine intelligence, Vol. 43, 5 (2019), 1666-1677."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2019.02.006"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-49257-7_15"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.14778\/3750601.3750700"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2024.3397508"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1963405.1963487"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476311.3476407"},{"key":"e_1_2_1_8_1","volume-title":"IVFFLAT: Inverted file with exact post-verification. https:\/\/github.com\/facebookresearch\/faiss\/wiki\/Faiss-indexes. 2025\/10\/18.","author":"FAISS.","year":"2025","unstructured":"FAISS. 2025. IVFFLAT: Inverted file with exact post-verification. https:\/\/github.com\/facebookresearch\/faiss\/wiki\/Faiss-indexes. 2025\/10\/18."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.14778\/3681954.3681960"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.14778\/3748191.3748211"},{"key":"e_1_2_1_11_1","volume-title":"Fast approximate nearest neighbor search with the navigating spreading-out graph. arXiv preprint arXiv:1707.00143","author":"Fu Cong","year":"2017","unstructured":"Cong Fu, Chao Xiang, Changxu Wang, and Deng Cai. 2017. Fast approximate nearest neighbor search with the navigating spreading-out graph. arXiv preprint arXiv:1707.00143 (2017)."},{"key":"e_1_2_1_12_1","first-page":"1","article-title":"Two sample testing in high dimension via maximum mean discrepancy","volume":"24","author":"Gao Hanjia","year":"2023","unstructured":"Hanjia Gao and Xiaofeng Shao. 2023. Two sample testing in high dimension via maximum mean discrepancy. Journal of Machine Learning Research, Vol. 24, 304 (2023), 1-33.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3543507.3583552"},{"key":"e_1_2_1_14_1","volume-title":"Generative adversarial nets. Advances in neural information processing systems","author":"Goodfellow Ian J","year":"2014","unstructured":"Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Advances in neural information processing systems, Vol. 27 (2014)."},{"key":"e_1_2_1_15_1","first-page":"203","volume-title":"Foundations and Trends\u00ae in Databases","volume":"3","author":"Goetz","year":"2011","unstructured":"Goetz Graefe et al., 2011. Modern B-tree techniques. Foundations and Trends\u00ae in Databases, Vol. 3, 4 (2011), 203-402."},{"key":"e_1_2_1_16_1","volume-title":"A kernel two-sample test. The journal of machine learning research","author":"Gretton Arthur","year":"2012","unstructured":"Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Sch\u00f6lkopf, and Alexander Smola. 2012. A kernel two-sample test. The journal of machine learning research, Vol. 13, 1 (2012), 723-773."},{"key":"e_1_2_1_17_1","first-page":"141","volume-title":"Foundations and Trends\u00ae in Databases","volume":"1","author":"Hellerstein Joseph M","year":"2007","unstructured":"Joseph M Hellerstein, Michael Stonebraker, James Hamilton, et al., 2007. Architecture of a database system. Foundations and Trends\u00ae in Databases, Vol. 1, 2 (2007), 141-259."},{"key":"e_1_2_1_18_1","volume-title":"Denoising diffusion probabilistic models. Advances in neural information processing systems","author":"Ho Jonathan","year":"2020","unstructured":"Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, Vol. 33 (2020), 6840-6851."},{"key":"e_1_2_1_19_1","volume-title":"Product quantization for nearest neighbor search","author":"Jegou Herve","year":"2010","unstructured":"Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2010. Product quantization for nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence, Vol. 33, 1 (2010), 117-128."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2011.5946540"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBDATA.2019.2921572"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.14778\/3718057.3718078"},{"key":"e_1_2_1_23_1","volume-title":"Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114","author":"Kingma Diederik P","year":"2013","unstructured":"Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)."},{"key":"e_1_2_1_24_1","first-page":"82","article-title":"The Johnson-Lindenstrauss Lemma Is Optimal for Linear Dimensionality Reduction","volume":"55","author":"Larsen Kasper Green","year":"2016","unstructured":"Kasper Green Larsen and Jelani Nelson. 2016. The Johnson-Lindenstrauss Lemma Is Optimal for Linear Dimensionality Reduction. Leibniz International Proceedings in Informatics, Vol. 55 (2016), 82-1.","journal-title":"Leibniz International Proceedings in Informatics"},{"key":"e_1_2_1_25_1","volume-title":"Adaptive estimation of a quadratic functional by model selection. Annals of statistics","author":"Laurent Beatrice","year":"2000","unstructured":"Beatrice Laurent and Pascal Massart. 2000. Adaptive estimation of a quadratic functional by model selection. Annals of statistics (2000), 1302-1338."},{"key":"e_1_2_1_26_1","volume-title":"UNIFY: Unified Index for Range Filtered Approximate Nearest Neighbors Search. arXiv preprint arXiv:2412.02448","author":"Liang Anqi","year":"2024","unstructured":"Anqi Liang, Pengcheng Zhang, Bin Yao, Zhongpu Chen, Yitong Song, and Guangxu Cheng. 2024. UNIFY: Unified Index for Range Filtered Approximate Nearest Neighbors Search. arXiv preprint arXiv:2412.02448 (2024)."},{"key":"e_1_2_1_27_1","volume-title":"Fast Vector Search in PostgreSQL: A Decoupled Approach. In Conference on Innovative Data Systems Research (CIDR).","author":"Liu Jiayi","year":"2026","unstructured":"Jiayi Liu, Yunan Zhang, Chenzhe Jin, Aditya Gupta, Shige Liu, and Jianguo Wang. 2026. Fast Vector Search in PostgreSQL: A Decoupled Approach. In Conference on Innovative Data Systems Research (CIDR)."},{"key":"e_1_2_1_28_1","volume-title":"A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going? IEEE Transactions on Knowledge and Data Engineering","author":"Liu Xinyu","year":"2025","unstructured":"Xinyu Liu, Shuyu Shen, Boyan Li, Peixian Ma, Runzhi Jiang, Yuxin Zhang, Ju Fan, Guoliang Li, Nan Tang, and Yuyu Luo. 2025. A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going? IEEE Transactions on Knowledge and Data Engineering (2025)."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2013.10.006"},{"key":"e_1_2_1_30_1","volume-title":"Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs","author":"Malkov Yu A","year":"2018","unstructured":"Yu A Malkov and Dmitry A Yashunin. 2018. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE transactions on pattern analysis and machine intelligence, Vol. 42, 4 (2018), 824-836."},{"key":"e_1_2_1_31_1","volume-title":"Damian Borth, Barry Chen, and Eric Wang.","author":"Ni Karl","year":"2015","unstructured":"Karl Ni, Roger Pearce, Kofi Boakye, Brian Van Essen, Damian Borth, Barry Chen, and Eric Wang. 2015. Large-scale deep learning on the YFCC100M dataset. arXiv preprint arXiv:1502.03409 (2015)."},{"key":"e_1_2_1_32_1","unstructured":"Openai. 2025. ChatGPT Retrieval Plugin. https:\/\/github.com\/openai\/chatgpt-retrieval-plugin\/. 2025\/10\/18."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.14778\/3750601.3750703"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-024-00864-x"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626246.3654691"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3654923"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3415148"},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the VLDB Endowment","author":"Qin Jianbin","year":"2020","unstructured":"Jianbin Qin, Wei Wang, Chuan Xiao, and Ying Zhang. 2020. Similarity query processing for high-dimensional data. Proceedings of the VLDB Endowment (2020)."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.14778\/3685800.3685802"},{"key":"e_1_2_1_40_1","volume-title":"Dmitry Baranchuk, Edo Liberty, Frank Liu, Ben Landrum, et al.","author":"Simhadri Harsha Vardhan","year":"2024","unstructured":"Harsha Vardhan Simhadri, Martin Aum\u00fcller, Amir Ingber, Matthijs Douze, George Williams, Magdalen Dobson Manohar, Dmitry Baranchuk, Edo Liberty, Frank Liu, Ben Landrum, et al., 2024. Results of the Big ANN: NeurIPS'23 competition. arXiv preprint arXiv:2409.17424 (2024)."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.14778\/3750601.3750619"},{"key":"e_1_2_1_42_1","volume-title":"The random projection method","author":"Vempala Santosh S","unstructured":"Santosh S Vempala. 2005. The random projection method. Vol. 65. American Mathematical Soc."},{"key":"e_1_2_1_43_1","first-page":"15738","article-title":"An efficient and robust framework for approximate nearest neighbor search with attribute constraint","volume":"36","author":"Wang Mengzhao","year":"2023","unstructured":"Mengzhao Wang, Lingwei Lv, Xiaoliang Xu, Yuxiang Wang, Qiang Yue, and Jiongkang Ni. 2023. An efficient and robust framework for approximate nearest neighbor search with attribute constraint. Advances in Neural Information Processing Systems, Vol. 36 (2023), 15738-15751.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415541"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.14778\/3067421.3067427"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3386131"},{"key":"e_1_2_1_47_1","unstructured":"Chao Zhang et al. 2018. Parameter Curation and Data Generation for Benchmarking Multi-model Queries. PhD@ VLDB (2018)."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE65448.2025.00191"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2024.3389693"},{"key":"e_1_2_1_50_1","volume-title":"Deepanalyze: Agentic large language models for autonomous data science. arXiv preprint arXiv:2510.16872","author":"Zhang Shaolei","year":"2025","unstructured":"Shaolei Zhang, Ju Fan, Meihao Fan, Guoliang Li, and Xiaoyong Du. 2025a. Deepanalyze: Agentic large language models for autonomous data science. arXiv preprint arXiv:2510.16872 (2025)."},{"key":"e_1_2_1_51_1","volume-title":"Reward-sql: Boosting text-to-sql via stepwise reasoning and process-supervised rewards. arXiv preprint arXiv:2505.04671","author":"Zhang Yuxin","year":"2025","unstructured":"Yuxin Zhang, Meihao Fan, Ju Fan, Mingyang Yi, Yuyu Luo, Jian Tan, and Guoliang Li. 2025b. Reward-sql: Boosting text-to-sql via stepwise reasoning and process-supervised rewards. arXiv preprint arXiv:2505.04671 (2025)."},{"key":"e_1_2_1_52_1","volume-title":"2024 IEEE 40th International Conference on Data Engineering (ICDE). IEEE, 3640-3653","author":"Zhang Yunan","year":"2024","unstructured":"Yunan Zhang, Shige Liu, and Jianguo Wang. 2024b. Are there fundamental limitations in supporting vector data management in relational databases? A case study of PostgreSQL. In 2024 IEEE 40th International Conference on Data Engineering (ICDE). IEEE, 3640-3653."},{"key":"e_1_2_1_53_1","unstructured":"Zilliz. 2025. VectorDBBench: A Benchmark Tool for VectorDB. https:\/\/github.com\/zilliztech\/VectorDBBench. 2025\/10\/12."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3639324"}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3802125","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T18:32:11Z","timestamp":1779129131000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3802125"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,18]]},"references-count":54,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,5,18]]}},"alternative-id":["10.1145\/3802125"],"URL":"https:\/\/doi.org\/10.1145\/3802125","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,18]]}}}