{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T10:19:48Z","timestamp":1782469188724,"version":"3.54.5"},"reference-count":57,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,8]]},"abstract":"<jats:p>Vector databases have recently gained significant attention due to the emergence of large language models that produce vector embeddings for text. Existing vector databases can be broadly categorized into two types: specialized and generalized. Specialized vector databases are explicitly designed and optimized for managing vector data, while generalized ones support vector data management within a general purpose database. While specialized vector databases are interesting, there is a substantial customer base interested in generalized vector databases for various reasons, e.g., a reluctance to move data out of relational databases to reduce data silos and costs, the desire to use SQL, and the need for more sophisticated query processing of vector and non-vector data. However, generalized vector databases face two main challenges: performance and interoperability of vector search with SQL, such as combining vector search with filters, joins, or even fulltext search.<\/jats:p>\n          <jats:p>In this paper, we present SingleStore-V, a full-fledged generalized vector database integrated into SingleStore, a modern distributed relational database optimized for both OLAP and OLTP workloads. SingleStore-V achieves high performance and interoperability via a suite of optimizations. Experiments on standard vector benchmarks show that SingleStore-V performs comparably to Milvus, a highly-optimized specialized vector database, and significantly outperforms pgvector, a popular generalized vector database in PostgreSQL. We believe this paper will shed light on integrating vector search into relational databases in general, as many design concepts and optimizations apply to other databases.<\/jats:p>","DOI":"10.14778\/3685800.3685805","type":"journal-article","created":{"date-parts":[[2024,11,8]],"date-time":"2024-11-08T17:25:21Z","timestamp":1731086721000},"page":"3772-3785","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":24,"title":["SingleStore-V: An Integrated Vector Database System in SingleStore"],"prefix":"10.14778","volume":"17","author":[{"given":"Cheng","family":"Chen","sequence":"first","affiliation":[{"name":"SingleStore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chenzhe","family":"Jin","sequence":"additional","affiliation":[{"name":"Purdue University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yunan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Purdue University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sasha","family":"Podolsky","sequence":"additional","affiliation":[{"name":"SingleStore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chun","family":"Wu","sequence":"additional","affiliation":[{"name":"SingleStore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Szu-Po","family":"Wang","sequence":"additional","affiliation":[{"name":"SingleStore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eric","family":"Hanson","sequence":"additional","affiliation":[{"name":"SingleStore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhou","family":"Sun","sequence":"additional","affiliation":[{"name":"SingleStore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"Walzer","sequence":"additional","affiliation":[{"name":"SingleStore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianguo","family":"Wang","sequence":"additional","affiliation":[{"name":"Purdue University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,11,8]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"[n.d.]. AutoFaiss (https:\/\/github.com\/criteo\/autofaiss)."},{"key":"e_1_2_1_2_1","unstructured":"[n. d.]. ClickHouse Approximate Nearest Neighbor Search Indexes [Experimental] (https:\/\/clickhouse.com\/docs\/en\/engines\/table-engines\/mergetree-family\/annindexes)."},{"key":"e_1_2_1_3_1","unstructured":"[n. d.]. Cohere (https:\/\/huggingface.co\/datasets\/Cohere\/wikipedia-22-12\/tree\/main\/en\/)."},{"key":"e_1_2_1_4_1","unstructured":"[n.d.]. ElasticSearch Github (https:\/\/github.com\/elastic\/elasticsearch)."},{"key":"e_1_2_1_5_1","unstructured":"[n.d.]. Facebook Faiss. https:\/\/github.com\/facebookresearch\/faiss"},{"key":"e_1_2_1_6_1","unstructured":"[n. d.]. Faiss AutoTune (https:\/\/github.com\/facebookresearch\/faiss\/wiki\/Index-IO -cloning-and-hyper-parameter-tuning#auto-tuning-the-runtimeparameters)."},{"key":"e_1_2_1_7_1","unstructured":"[n. d.]. Faiss Indexes. https:\/\/github.com\/facebookresearch\/faiss\/wiki\/Faiss-indexes"},{"key":"e_1_2_1_8_1","unstructured":"[n.d.]. Fast Accumulation of PQ and AQ Codes (FastScan) (https:\/\/github.com\/facebookresearch\/faiss\/wiki\/Fast-accumulation-of-PQ-and-AQ-codes-(FastScan))."},{"key":"e_1_2_1_9_1","unstructured":"[n. d.]. Guidelines to Choose an Index in Faiss (https:\/\/github.com\/facebookresearch\/faiss\/wiki\/Guidelines-to-choose-an-index)."},{"key":"e_1_2_1_10_1","unstructured":"[n. d.]. Introducing Vector Search on Rockset: How to Run Semantic Search with OpenAI and Rockset (https:\/\/rockset.com\/blog\/introducing-vector-search-on-rockset\/)."},{"key":"e_1_2_1_11_1","unstructured":"[n. d.]. Milvus AUTOINDEX Explained (https:\/\/docs.zilliz.com\/docs\/autoindex-explained)."},{"key":"e_1_2_1_12_1","unstructured":"[n. d.]. Milvus Knowhere (https:\/\/milvus.io\/docs\/knowhere.md)."},{"key":"e_1_2_1_13_1","unstructured":"[n. d.]. MongoDB (https:\/\/www.mongodb.com)."},{"key":"e_1_2_1_14_1","unstructured":"[n. d.]. MongoDB Vector Search (https:\/\/www.mongodb.com\/products\/platform\/atlas-vector-search)."},{"key":"e_1_2_1_15_1","unstructured":"[n. d.]. pgvector (https:\/\/github.com\/pgvector\/pgvector)."},{"key":"e_1_2_1_16_1","unstructured":"[n. d.]. Pinecone (https:\/\/www.pinecone.io\/)."},{"key":"e_1_2_1_17_1","unstructured":"[n. d.]. Qdrant (https:\/\/qdrant.tech\/)."},{"key":"e_1_2_1_18_1","unstructured":"[n. d.]. Recommender System in Milvus (https:\/\/milvus.io\/docs\/recommendation_system.md)."},{"key":"e_1_2_1_19_1","unstructured":"[n. d.]. SIFTData (http:\/\/corpus-texmex.irisa.fr\/)."},{"key":"e_1_2_1_20_1","unstructured":"[n. d.]. VectorDBBench: A Benchmark Tool for VectorDB (https:\/\/github.com\/zilliztech\/VectorDBBench)."},{"key":"e_1_2_1_21_1","unstructured":"[n. d.]. Vespa (https:\/\/vespa.ai\/)."},{"key":"e_1_2_1_22_1","unstructured":"[n. d.]. Weaviate (https:\/\/weaviate.io\/)."},{"key":"e_1_2_1_23_1","unstructured":"2023. Oracle Introduces Integrated Vector Database to Augment Generative AI and Dramatically Increase Developer Productivity (https:\/\/www.oracle.com\/news\/announcement\/ocw-integrated-vector-database-augments-generative-ai-2023-09-19\/)."},{"key":"e_1_2_1_24_1","unstructured":"2023. Relevance Scoring in Hybrid Search using Reciprocal Rank Fusion (RRF) (https:\/\/learn.microsoft.com\/en-us\/azure\/search\/hybrid-search-ranking)."},{"key":"e_1_2_1_25_1","unstructured":"2023. Vector Search with Azure SQL Database (https:\/\/devblogs.microsoft.com\/azure-sql\/vector-search-with-azure-sql-database\/)."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.14778\/3007263.3007277"},{"key":"e_1_2_1_27_1","volume-title":"SPANN: Highly-efficient Billion-scale Approximate Nearest Neighborhood Search. In Annual Conference on Neural Information Processing Systems (NeurIPS). 5199--5212","author":"Chen Qi","year":"2021","unstructured":"Qi Chen, Bing Zhao, Haidong Wang, Mingqin Li, Chuanjie Liu, Zengzhong Li, Mao Yang, and Jingdong Wang. 2021. SPANN: Highly-efficient Billion-scale Approximate Nearest Neighborhood Search. In Annual Conference on Neural Information Processing Systems (NeurIPS). 5199--5212."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-2614"},{"key":"e_1_2_1_29_1","volume-title":"Conference on Innovative Data Systems Research (CIDR). 195--198","author":"Dittrich Jens","year":"2011","unstructured":"Jens Dittrich and Alekh Jindal. 2011. Towards a One Size Fits All Database Architecture. In Conference on Innovative Data Systems Research (CIDR). 195--198."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.14778\/3303753.3303754"},{"key":"e_1_2_1_31_1","volume-title":"International Conference on Very Large Data Bases (VLDB). 518--529","author":"Gionis Aristides","year":"1999","unstructured":"Aristides Gionis, Piotr Indyk, and Rajeev Motwani. 1999. Similarity Search in High Dimensions via Hashing. In International Conference on Very Large Data Bases (VLDB). 518--529."},{"key":"e_1_2_1_32_1","unstructured":"Samuel Leonardo Gracio. 2023. Reinvent your recommender system using Vector Database and Opinion Mining (https:\/\/medium.com\/dailymotion\/reinvent-your-recommender-system-using-vector-database-and-opinion-mining-a4fadf97d020)."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.57"},{"key":"e_1_2_1_34_1","unstructured":"Madhukar Kumar Yaroslav Demenskyi and Pranav Aurora. 2024. How We Built a Real-Time RAG Application for Free With SingleStore and Vercel (https:\/\/www.singlestore.com\/blog\/real-time-rag-app-with-singlestore-and-vercel\/)."},{"key":"e_1_2_1_35_1","volume-title":"LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In International Symposium on Code Generation and Optimization (CGO). 75--88","author":"Lattner Chris","unstructured":"Chris Lattner and Vikram S. Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In International Symposium on Code Generation and Optimization (CGO). 75--88."},{"key":"e_1_2_1_36_1","volume-title":"Cosine Threshold Querying with Optimality Guarantees. In International Conference on Database Theory (ICDT)","volume":"127","author":"Li Yuliang","year":"2019","unstructured":"Yuliang Li, Jianguo Wang, Benjamin S. Pullman, Nuno Bandeira, and Yannis Papakonstantinou. 2019. Index-Based, High-Dimensional, Cosine Threshold Querying with Optimality Guarantees. In International Conference on Database Theory (ICDT), Vol. 127. 11:1--11:20."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.14778\/3397230.3397240"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137765.3137836"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2018.2889473"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3190658"},{"key":"e_1_2_1_41_1","volume-title":"Survey of Vector Database Management Systems. CoRR abs\/2310.14021","author":"Pan James Jie","year":"2023","unstructured":"James Jie Pan, Jianguo Wang, and Guoliang Li. 2023. Survey of Vector Database Management Systems. CoRR abs\/2310.14021 (2023)."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626246.3654691"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3526055"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2016.7498332"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2005.1"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3654990"},{"key":"e_1_2_1_47_1","volume-title":"Annual Conference on Neural Information Processing Systems (NeurIPS). 13748--13758","author":"Subramanya Suhas Jayaram","year":"2019","unstructured":"Suhas Jayaram Subramanya, Fnu Devvrit, Harsha Vardhan Simhadri, Ravishankar Krishnaswamy, and Rohan Kadekodi. 2019. Rand-NSG: Fast Accurate Billionpoint Nearest Neighbor Search on a Single Node. In Annual Conference on Neural Information Processing Systems (NeurIPS). 13748--13758."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3097987"},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of the VLDB Endowment (PVLDB) 17","author":"Wang Jianguo","year":"2024","unstructured":"Jianguo Wang, Eric Hanson, Guoliang Li, Yannis Papakonstantinou, Harsha Simhadri, and Charles Xie. 2024. Vector Databases: What's Really New and What's Next? Proceedings of the VLDB Endowment (PVLDB) 17 (2024)."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457550"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.14778\/3424573.3424580"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415541"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3386131"},{"key":"e_1_2_1_54_1","unstructured":"Alan Zeichick. 2023. What Is Retrieval-Augmented Generation (RAG)? (https:\/\/www.oracle.com\/artificial-intelligence\/generative-ai\/retrieval-augmented-generation-rag\/)."},{"key":"e_1_2_1_55_1","volume-title":"USENIX Symposium on Operating Systems Design and Implementation (OSDI). 377--395","author":"Zhang Qianxi","year":"2023","unstructured":"Qianxi Zhang, Shuotao Xu, Qi Chen, Guoxin Sui, Jiadong Xie, Zhizhen Cai, Yaoqi Chen, Yinxuan He, Yuqing Yang, Fan Yang, Mao Yang, and Lidong Zhou. 2023. VBASE: Unifying Online Vector Similarity Search and Relational Queries via Relaxed Monotonicity. In USENIX Symposium on Operating Systems Design and Implementation (OSDI). 377--395."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE60146.2024.00280"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.14778\/3377369.3377374"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3685800.3685805","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,31]],"date-time":"2024-12-31T05:25:01Z","timestamp":1735622701000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3685800.3685805"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8]]},"references-count":57,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2024,8]]}},"alternative-id":["10.14778\/3685800.3685805"],"URL":"https:\/\/doi.org\/10.14778\/3685800.3685805","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,8]]},"assertion":[{"value":"2024-11-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}