{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T09:59:27Z","timestamp":1773482367957,"version":"3.50.1"},"reference-count":44,"publisher":"Association for Computing Machinery (ACM)","issue":"9","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2023,5]]},"abstract":"<jats:p>State-of-the-art video database management systems (VDBMSs) often use lightweight proxy models to accelerate object retrieval and aggregate queries. The key assumption underlying these systems is that the proxy model is an order of magnitude faster than the heavyweight oracle model. However, recent advances in computer vision have invalidated this assumption. Inference time of recently proposed oracle models is on par with or even lower than the proxy models used in state-of-the-art (SoTA) VDBMSs. This paper presents Seiden, a VDBMS that leverages this radical shift in the runtime gap between the oracle and proxy models. Instead of relying on a proxy model, Seiden directly applies the oracle model over a subset of frames to build a query-agnostic index, and samples additional frames to answer the query using an exploration-exploitation scheme during query processing. By leveraging the temporal continuity of the video and the output of the oracle model on the sampled frames, Seiden delivers faster query processing and better query accuracy than SoTA VDBMSs. Our empirical evaluation shows that Seiden is on average 6.6 x faster than SoTA VDBMSs across diverse queries and datasets.<\/jats:p>","DOI":"10.14778\/3598581.3598599","type":"journal-article","created":{"date-parts":[[2023,7,10]],"date-time":"2023-07-10T22:19:06Z","timestamp":1689027546000},"page":"2289-2301","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Seiden: Revisiting Query Processing in Video Database Systems"],"prefix":"10.14778","volume":"16","author":[{"given":"Jaeho","family":"Bang","sequence":"first","affiliation":[{"name":"Georgia Institute of Technology"}]},{"given":"Gaurav Tarlok","family":"Kakkar","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology"}]},{"given":"Pramod","family":"Chunduri","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology"}]},{"given":"Subrata","family":"Mitra","sequence":"additional","affiliation":[{"name":"Adobe Research"}]},{"given":"Joy","family":"Arulraj","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology"}]}],"member":"320","published-online":{"date-parts":[[2023,7,10]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2465351.2465355"},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1080\/00031305.1992.10475879","article-title":"An introduction to kernel and nearest-neighbor nonparametric regression","volume":"46","author":"Altman Naomi S","year":"1992","unstructured":"Naomi S Altman . 1992 . An introduction to kernel and nearest-neighbor nonparametric regression . The American Statistician 46 , 3 (1992), 175 -- 185 . Naomi S Altman. 1992. An introduction to kernel and nearest-neighbor nonparametric regression. The American Statistician 46, 3 (1992), 175--185.","journal-title":"The American Statistician"},{"key":"e_1_2_1_3_1","volume-title":"Physical representation-based predicate optimization for a visual analytics database","author":"Anderson Michael R","unstructured":"Michael R Anderson , Michael Cafarella , German Ros , and Thomas F Wenisch . 2019. Physical representation-based predicate optimization for a visual analytics database . In ICDE. IEEE , 1466--1477. Michael R Anderson, Michael Cafarella, German Ros, and Thomas F Wenisch. 2019. Physical representation-based predicate optimization for a visual analytics database. In ICDE. IEEE, 1466--1477."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10590-1_38"},{"key":"e_1_2_1_5_1","volume-title":"Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques","author":"Braverman Vladimir","unstructured":"Vladimir Braverman and Rafail Ostrovsky . 2013. Generalizing the layering method of indyk and woodruff: Recursive sketches for frequency-based vectors on streams . In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques . Springer , 58--70. Vladimir Braverman and Rafail Ostrovsky. 2013. Generalizing the layering method of indyk and woodruff: Recursive sketches for frequency-based vectors on streams. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques. Springer, 58--70."},{"key":"e_1_2_1_6_1","doi-asserted-by":"crossref","unstructured":"S\u00e9bastien Bubeck Nicolo Cesa-Bianchi etal 2012. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends\u00ae in Machine Learning 5 1 (2012) 1--122. S\u00e9bastien Bubeck Nicolo Cesa-Bianchi et al. 2012. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends\u00ae in Machine Learning 5 1 (2012) 1--122.","DOI":"10.1561\/2200000024"},{"key":"e_1_2_1_7_1","volume-title":"Radial basis functions. Acta numerica 9","author":"Buhmann Martin D","year":"2000","unstructured":"Martin D Buhmann . 2000. Radial basis functions. Acta numerica 9 ( 2000 ), 1--38. Martin D Buhmann. 2000. Radial basis functions. Acta numerica 9 (2000), 1--38."},{"key":"e_1_2_1_8_1","volume-title":"ECML PKDD Workshop: Languages for Data Mining and Machine Learning. 108--122","author":"Buitinck Lars","year":"2013","unstructured":"Lars Buitinck , Gilles Louppe , Mathieu Blondel , Fabian Pedregosa , Andreas Mueller , Olivier Grisel , Vlad Niculae , Peter Prettenhofer , Alexandre Gramfort , Jaques Grobler , Robert Layton , Jake VanderPlas , Arnaud Joly , Brian Holt , and Ga\u00ebl Varoquaux . 2013 . API design for machine learning software: experiences from the scikit-learn project . In ECML PKDD Workshop: Languages for Data Mining and Machine Learning. 108--122 . Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake VanderPlas, Arnaud Joly, Brian Holt, and Ga\u00ebl Varoquaux. 2013. API design for machine learning software: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning. 108--122."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3517857"},{"key":"e_1_2_1_10_1","volume-title":"Zeus: Efficiently Localizing Actions in Videos Using Reinforcement Learning. In SIGMOD. 545--558.","author":"Chunduri Pramod","year":"2022","unstructured":"Pramod Chunduri , Jaeho Bang , Yao Lu , and Joy Arulraj . 2022 . Zeus: Efficiently Localizing Actions in Videos Using Reinforcement Learning. In SIGMOD. 545--558. Pramod Chunduri, Jaeho Bang, Yao Lu, and Joy Arulraj. 2022. Zeus: Efficiently Localizing Actions in Videos Using Reinforcement Learning. In SIGMOD. 545--558."},{"key":"e_1_2_1_11_1","volume-title":"TASM: A Tile-Based Storage Manager for Video Analytics. arXiv preprint arXiv:2006.02958","author":"Daum Maureen","year":"2020","unstructured":"Maureen Daum , Brandon Haynes , Dong He , Amrita Mazumdar , Magdalena Balazinska , and Alvin Cheung . 2020 . TASM: A Tile-Based Storage Manager for Video Analytics. arXiv preprint arXiv:2006.02958 (2020). Maureen Daum, Brandon Haynes, Dong He, Amrita Mazumdar, Magdalena Balazinska, and Alvin Cheung. 2020. TASM: A Tile-Based Storage Manager for Video Analytics. arXiv preprint arXiv:2006.02958 (2020)."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995516"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i6.25929"},{"key":"e_1_2_1_14_1","volume-title":"Clustering to minimize the maximum intercluster distance. Theoretical computer science 38","author":"Gonzalez Teofilo F","year":"1985","unstructured":"Teofilo F Gonzalez . 1985. Clustering to minimize the maximum intercluster distance. Theoretical computer science 38 ( 1985 ), 293--306. Teofilo F Gonzalez. 1985. Clustering to minimize the maximum intercluster distance. Theoretical computer science 38 (1985), 293--306."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1081870.1081884"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.14778\/3231751.3231768"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.322"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389766"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/253260.253291"},{"key":"e_1_2_1_21_1","volume-title":"Focus: Querying large video datasets with low latency and low cost. In USENIX. 269--286.","author":"Hsieh Kevin","year":"2018","unstructured":"Kevin Hsieh , Ganesh Ananthanarayanan , Peter Bodik , Shivaram Venkataraman , Paramvir Bahl , Matthai Philipose , Phillip B Gibbons , and Onur Mutlu . 2018 . Focus: Querying large video datasets with low latency and low cost. In USENIX. 269--286. Kevin Hsieh, Ganesh Ananthanarayanan, Peter Bodik, Shivaram Venkataraman, Paramvir Bahl, Matthai Philipose, Phillip B Gibbons, and Onur Mutlu. 2018. Focus: Querying large video datasets with low latency and low cost. In USENIX. 269--286."},{"key":"e_1_2_1_22_1","unstructured":"Daniel Kang. 2023. TASTI. https:\/\/github.com\/ddkang\/tasti Daniel Kang. 2023. TASTI. https:\/\/github.com\/ddkang\/tasti"},{"key":"e_1_2_1_23_1","volume-title":"Blazeit: Optimizing declarative aggregation and limit queries for neural network-based video analytics. arXiv preprint arXiv:1805.01046","author":"Kang Daniel","year":"2018","unstructured":"Daniel Kang , Peter Bailis , and Matei Zaharia . 2018 . Blazeit: Optimizing declarative aggregation and limit queries for neural network-based video analytics. arXiv preprint arXiv:1805.01046 (2018). Daniel Kang, Peter Bailis, and Matei Zaharia. 2018. Blazeit: Optimizing declarative aggregation and limit queries for neural network-based video analytics. arXiv preprint arXiv:1805.01046 (2018)."},{"key":"e_1_2_1_24_1","volume-title":"Noscope: optimizing neural network queries over video at scale. arXiv preprint arXiv:1703.02529","author":"Kang Daniel","year":"2017","unstructured":"Daniel Kang , John Emmons , Firas Abuzaid , Peter Bailis , and Matei Zaharia . 2017. Noscope: optimizing neural network queries over video at scale. arXiv preprint arXiv:1703.02529 ( 2017 ). Daniel Kang, John Emmons, Firas Abuzaid, Peter Bailis, and Matei Zaharia. 2017. Noscope: optimizing neural network queries over video at scale. arXiv preprint arXiv:1703.02529 (2017)."},{"key":"e_1_2_1_25_1","volume-title":"Approximate selection with guarantees using proxies. arXiv preprint arXiv:2004.00827","author":"Kang Daniel","year":"2020","unstructured":"Daniel Kang , Edward Gan , Peter Bailis , Tatsunori Hashimoto , and Matei Zaharia . 2020. Approximate selection with guarantees using proxies. arXiv preprint arXiv:2004.00827 ( 2020 ). Daniel Kang, Edward Gan, Peter Bailis, Tatsunori Hashimoto, and Matei Zaharia. 2020. Approximate selection with guarantees using proxies. arXiv preprint arXiv:2004.00827 (2020)."},{"key":"e_1_2_1_26_1","volume-title":"Task-agnostic Indexes for Deep Learning-based Queries over Unstructured Data. arXiv preprint arXiv:2009.04540","author":"Kang Daniel","year":"2020","unstructured":"Daniel Kang , John Guibas , Peter Bailis , Tatsunori Hashimoto , and Matei Zaharia . 2020. Task-agnostic Indexes for Deep Learning-based Queries over Unstructured Data. arXiv preprint arXiv:2009.04540 ( 2020 ). Daniel Kang, John Guibas, Peter Bailis, Tatsunori Hashimoto, and Matei Zaharia. 2020. Task-agnostic Indexes for Deep Learning-based Queries over Unstructured Data. arXiv preprint arXiv:2009.04540 (2020)."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/s41019-018-0074-4"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2015.7301269"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"e_1_2_1_30_1","volume-title":"Ssd: Single shot multibox detector","author":"Liu Wei","year":"2016","unstructured":"Wei Liu , Dragomir Anguelov , Dumitru Erhan , Christian Szegedy , Scott Reed , Cheng-Yang Fu , and Alexander C Berg . 2016 . Ssd: Single shot multibox detector . In ECCV. Springer , 21--37. Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. 2016. Ssd: Single shot multibox detector. In ECCV. Springer, 21--37."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3183751"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3324958"},{"key":"e_1_2_1_33_1","unstructured":"Seattle Department of Transportation. 2023. Travelers Home Page. https:\/\/web6.seattle.gov\/travelers\/ Seattle Department of Transportation. 2023. Travelers Home Page. https:\/\/web6.seattle.gov\/travelers\/"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/971697.602294"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10619-013-7132-8"},{"key":"e_1_2_1_36_1","doi-asserted-by":"crossref","unstructured":"Joseph Redmon Santosh Divvala Ross Girshick and Ali Farhadi. 2016. You only look once: Unified real-time object detection. In CVPR. 779--788. Joseph Redmon Santosh Divvala Ross Girshick and Ali Farhadi. 2016. You only look once: Unified real-time object detection. In CVPR. 779--788.","DOI":"10.1109\/CVPR.2016.91"},{"key":"e_1_2_1_37_1","volume-title":"The H. 264 advanced video compression standard","author":"Richardson Iain E","unstructured":"Iain E Richardson . 2011. The H. 264 advanced video compression standard . John Wiley & Sons . Iain E Richardson. 2011. The H. 264 advanced video compression standard. John Wiley & Sons."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i8.20800"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01079"},{"key":"e_1_2_1_40_1","unstructured":"Ultralytics. 2023. YOLOv5. https:\/\/github.com\/ultralytics\/yolov5 Ultralytics. 2023. YOLOv5. https:\/\/github.com\/ultralytics\/yolov5"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2020.102907"},{"key":"e_1_2_1_42_1","volume-title":"Luis Materon Botelho, and Felix Xiaozhu Lin","author":"Xu Tiantu","year":"2019","unstructured":"Tiantu Xu , Luis Materon Botelho, and Felix Xiaozhu Lin . 2019 . Vstore : A data store for analytics on large videos. In EuroSys . 1--17. Tiantu Xu, Luis Materon Botelho, and Felix Xiaozhu Lin. 2019. Vstore: A data store for analytics on large videos. In EuroSys. 1--17."},{"key":"e_1_2_1_43_1","volume-title":"Joy Arulraj, and Umakishore Ramachandran.","author":"Xu Zhuangdi","year":"2022","unstructured":"Zhuangdi Xu , Gaurav Tarlok Kakkar , Joy Arulraj, and Umakishore Ramachandran. 2022 . EVA : A Symbolic Approach to Accelerating Exploratory Video Analytics with Materialized Views. In SIGMOD. 602--616. Zhuangdi Xu, Gaurav Tarlok Kakkar, Joy Arulraj, and Umakishore Ramachandran. 2022. EVA: A Symbolic Approach to Accelerating Exploratory Video Analytics with Materialized Views. In SIGMOD. 602--616."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.14778\/3372716.3372721"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3598581.3598599","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,23]],"date-time":"2024-10-23T22:48:13Z","timestamp":1729723693000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3598581.3598599"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5]]},"references-count":44,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2023,5]]}},"alternative-id":["10.14778\/3598581.3598599"],"URL":"https:\/\/doi.org\/10.14778\/3598581.3598599","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2023,5]]},"assertion":[{"value":"2023-07-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}