{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T06:33:13Z","timestamp":1790749993542,"version":"4.1.0"},"reference-count":40,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,5,29]],"date-time":"2024-05-29T00:00:00Z","timestamp":1716940800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100006374","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2239326"],"award-info":[{"award-number":["2239326"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2024,5,29]]},"abstract":"<jats:p>We introduce ThalamusDB, a novel approximate query processing system that processes complex SQL queries on multi-modal data. ThalamusDB supports SQL queries integrating natural language predicates on visual, audio, and text data. To answer such queries, ThalamusDB exploits a collection of zero-shot models in combination with relational processing. ThalamusDB utilizes deterministic approximate query processing, harnessing the relative efficiency of relational processing to mitigate the computational demands of machine learning inference. For evaluating a natural language predicate, ThalamusDB requests a small number of labels from users. User can specify their preferences on the performance objective regarding the three relevant metrics: approximation error, computation time, and labeling overheads. The ThalamusDB query optimizer chooses optimized plans according to user preferences, prioritizing data processing and requested labels to maximize impact. Experiments with several real-world data sets, taken from Craigslist, YouTube, and Netflix, show that ThalamusDB achieves an average speedup of 35.0x over MindsDB, an exact processing baseline, and outperforms ABAE, a sampling-based method, in 78.9% of cases.<\/jats:p>","DOI":"10.1145\/3654989","type":"journal-article","created":{"date-parts":[[2024,5,30]],"date-time":"2024-05-30T09:44:53Z","timestamp":1717062293000},"page":"1-26","source":"Crossref","is-referenced-by-count":24,"title":["ThalamusDB: Approximate Query Processing on Multi-Modal Data"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-3880-902X","authenticated-orcid":false,"given":"Saehan","family":"Jo","sequence":"first","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7203-2349","authenticated-orcid":false,"given":"Immanuel","family":"Trummer","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,5,30]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Swarup Acharya Phillip B Gibbons and Viswanath Poosala. 1999. Aqua: a fast decision support systems using approximate query answers. In VLDB. 754--757."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/342009.335450"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2465351.2465355"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/nnnnnnn.nnnnnnn"},{"key":"e_1_2_2_5_1","volume-title":"Advances in Neural Information Processing Systems. 1877--1901. arxiv","author":"Brown Tom B.","year":"2005","unstructured":"Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems. 1877--1901. arxiv: 2005.14165"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","unstructured":"Surajit Chaudhuri Bolin Ding and Srikanth Kandula. 2017. Approximate query processing : no silver bullet. In SIGMOD. 511--519. https:\/\/doi.org\/10.1145\/3035918.3056097","DOI":"10.1145\/3035918.3056097"},{"key":"e_1_2_2_7_1","volume-title":"Symphony: Towards Natural Language Query Answering over Multi-modal Data Lakes. In CIDR.","author":"Chen Zui","year":"2023","unstructured":"Zui Chen, Zihui Gu, Lei Cao, Ju Fan, Sam Madden, and Nan Tang. 2023. Symphony: Towards Natural Language Query Answering over Multi-modal Data Lakes. In CIDR."},{"key":"e_1_2_2_8_1","volume-title":"VOCAL : Video Organization and Interactive Compositional AnaLytics. In CIDR.","author":"Daum Maureen","year":"2022","unstructured":"Maureen Daum, Magdalena Balazinska, Brandon Haynes, Ranjay Krishna, Apryle Craig, and Aaron Wirsing. 2022. VOCAL : Video Organization and Interactive Compositional AnaLytics. In CIDR."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2206.04769"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2017.7952261"},{"key":"e_1_2_2_11_1","unstructured":"Brandon Haynes Maureen Daum Amrita Mazumdar Magdalena Balazinska Alvin Cheung and Luis Ceze. 2020. VisualWorldDB : A DBMS for the Visual World. In CIDR."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/253262.253291"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i14.17541"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","unstructured":"Jia Deng Wei Dong R. Socher Li-Jia Li Kai Li and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. (2009) 248--255. https:\/\/doi.org\/10.1109\/cvprw.2009.5206848","DOI":"10.1109\/cvprw.2009.5206848"},{"key":"e_1_2_2_15_1","unstructured":"Saehan Jo and Immanuel Trummer. 2020. BitGourmet: deterministic approximation via optimized bit selections. In CIDR. 1--7."},{"key":"e_1_2_2_16_1","unstructured":"Kaggle. 2023. Netflix Prize data. https:\/\/www.kaggle.com\/datasets\/netflix-inc\/netflix-prize-data."},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3595360.3595858"},{"key":"e_1_2_2_18_1","unstructured":"Daniel Kang. 2023. Accelerating Approximate Aggregation Queries with Expensive Predicates. https:\/\/github.com\/stanford-futuredata\/abae."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.14778\/3372716.3372725"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476249.3476285"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3517897"},{"key":"e_1_2_2_22_1","volume-title":"VIVA: An End-to-End System for Interactive Video Analytics","author":"Kang Daniel","year":"2022","unstructured":"Daniel Kang, Francisco Romero, Peter Bailis, Christos Kozyrakis, and Matei Zaharia. 2022b. VIVA: An End-to-End System for Interactive Video Analytics. In CIDR. Association for Computing Machinery."},{"key":"e_1_2_2_23_1","unstructured":"Chris Dongjoo Kim Byeongchang Kim Hyunmin Lee and Gunhee Kim. [n. d.]. AudioCaps: Generating Captions for Audios in The Wild."},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.12988\/ams.2013.13019"},{"key":"e_1_2_2_25_1","volume-title":"CIDR 2019 - 9th Biennial Conference on Innovative Data Systems Research (2019","author":"Krishnan Sanjay","year":"1812","unstructured":"Sanjay Krishnan, Adam Dziedzic, and Aaron J. Elmore. 2019. DeepLens: Towards a visual data management system. CIDR 2019 - 9th Biennial Conference on Innovative Data Systems Research (2019). arxiv: 1812.07607"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","unstructured":"Xi Liang Stavros Sintos Zechao Shang and Sanjay Krishnan. 2021. Combining Aggregation and Sampling (Nearly) Optimally for Approximate Query Processing. In SIGMOD. 1129--1141. https:\/\/doi.org\/10.1145\/3448016.3457277 arxiv: 2103.15994","DOI":"10.1145\/3448016.3457277"},{"key":"e_1_2_2_28_1","volume-title":"Learning Word Vectors for Sentiment Analysis. In The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, 19--24","author":"Maas Andrew L.","year":"2011","unstructured":"Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning Word Vectors for Sentiment Analysis. In The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, 19--24 June, 2011, Portland, Oregon, USA, Dekang Lin, Yuji Matsumoto, and Rada Mihalcea (Eds.). The Association for Computer Linguistics, 142--150. https:\/\/aclanthology.org\/P11--1015\/"},{"key":"e_1_2_2_29_1","doi-asserted-by":"crossref","unstructured":"Xinhao Mei Xubo Liu Jianyuan Sun Mark D. Plumbley and Wenwu Wang. 2022. On Metric Learning for Audio-Text Cross-Modal Retrieval. (2022). arxiv: 2203.15537 http:\/\/arxiv.org\/abs\/2203.15537","DOI":"10.21437\/Interspeech.2022-11115"},{"key":"e_1_2_2_30_1","unstructured":"MindsDB. 2023. MindsDB. https:\/\/mindsdb.com."},{"key":"e_1_2_2_31_1","volume-title":"SeeSaw: interactive ad-hoc search over image databases","author":"Moll Oscar","unstructured":"Oscar Moll, Manuel Favela, Samuel Madden, and Vijay Gadepally. 2022. SeeSaw: interactive ad-hoc search over image databases. In ArXiV. Association for Computing Machinery. arxiv: 2208.06497 http:\/\/arxiv.org\/abs\/2208.06497"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00140664"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","unstructured":"Gregory Piatetsky-Shapiro and Charles Connell. 1984. Accurate estimation of the number of tuples satisfying a condition. In SIGMOD. 256--276. https:\/\/doi.org\/10.1145\/602259.602294","DOI":"10.1145\/602259.602294"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.14778\/2777598.2777599"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3320212"},{"key":"e_1_2_2_36_1","volume-title":"Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. (2021). arxiv: 2103.00020 http:\/\/arxiv.org\/abs\/2103.00020"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/d19--1410"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00015"},{"key":"e_1_2_2_39_1","doi-asserted-by":"crossref","unstructured":"PG G Selinger MM M Astrahan D D Chamberlin R A Lorie and T G Price. 1979. Access path selection in a relational database management system. In SIGMOD. 23--34. http:\/\/dl.acm.org\/citation.cfm?id=582095.582099","DOI":"10.1145\/582095.582099"},{"key":"e_1_2_2_40_1","unstructured":"Yi Wang Yang Yang Weiguo Zhu Yi Wu Xu Yan Yongfeng Liu Yu Wang Liang Xie Ziyao Gao Wenjing Zhu et al. 2020. SQLflow: a bridge between SQL and machine learning. arXiv preprint arXiv:2001.06846 (2020)."}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654989","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3654989","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T14:38:34Z","timestamp":1755787114000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654989"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,29]]},"references-count":40,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,5,29]]}},"alternative-id":["10.1145\/3654989"],"URL":"https:\/\/doi.org\/10.1145\/3654989","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,29]]}}}