{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,19]],"date-time":"2026-08-19T20:18:00Z","timestamp":1787170680376,"version":"build-2736575974"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"4","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:p>With the exponential growth of video data, there is a pressing need for efficient video analysis technology. Modern query frameworks aim to accelerate queries by reducing the frequency of calls to expensive deep neural networks, which often overlook the overhead associated with video decoding and retrieval. Furthermore, video storage frameworks optimize video retrieval through video partition or caching, often relying on prior information about the query workload. To further accelerate queries, this study introduces a novel tile-based video management framework, called TVM, which leverages the semantic information embedded in videos, without being dependent on specific query workloads. By constructing a tile-based semantic index for newly ingested videos, TVM effectively reduces the size of decoded and processed video data. To achieve this, TVM introduces an optimal index construction algorithm that utilizes cost function and pseudo-labels. Additionally, the framework proposes a query-driven tile parallel decoding algorithm and resource caching algorithms, which further expedite the retrieval of video frames. Experimental results demonstrate that TVM can significantly enhance the throughput of various query tasks, achieving a notable speedup of more than 5.6\u00d7.<\/jats:p>","DOI":"10.14778\/3636218.3636224","type":"journal-article","created":{"date-parts":[[2024,3,5]],"date-time":"2024-03-05T12:04:07Z","timestamp":1709640247000},"page":"671-684","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["TVM: A Tile-based Video Management Framework"],"prefix":"10.14778","volume":"17","author":[{"given":"Tianxiong","family":"Zhong","sequence":"first","affiliation":[{"name":"Beijing Institute of Technology, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhiwei","family":"Zhang","sequence":"additional","affiliation":[{"name":"Beijing Institute of Technology, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guo","family":"Lu","sequence":"additional","affiliation":[{"name":"Shanghai Jiao Tong University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ye","family":"Yuan","sequence":"additional","affiliation":[{"name":"Beijing Institute of Technology, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu-Ping","family":"Wang","sequence":"additional","affiliation":[{"name":"Beijing Institute of Technology, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guoren","family":"Wang","sequence":"additional","affiliation":[{"name":"Beijing Institute of Technology, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,3,5]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2019.00132"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3356250.3360044"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2010.2101613"},{"key":"e_1_2_1_4_1","volume-title":"12th International Conference on Distributed Multimedia Systems (DMS","author":"Bartolini Ilaria","year":"2006","unstructured":"Ilaria Bartolini, Paolo Ciaccia, Lei Chen, Vincent Oria, et al. 2006. A meta-index to integrate specific indexes: application to multimedia. In 12th International Conference on Distributed Multimedia Systems (DMS 2006). Citeseer, 29--36."},{"key":"e_1_2_1_5_1","volume-title":"SHIATSU: tagging and retrieving videos without worries. Multimedia tools and applications 63","author":"Bartolini Ilaria","year":"2013","unstructured":"Ilaria Bartolini, Marco Patella, and Corrado Romani. 2013. SHIATSU: tagging and retrieving videos without worries. Multimedia tools and applications 63 (2013), 357--385."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355398"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389692"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 2022 International Conference on Management of Data. 2091--2104","author":"Bastani Favyen","year":"2022","unstructured":"Favyen Bastani and Samuel Madden. 2022. OTIF: Efficient tracker pre-processing over large video datasets. In Proceedings of the 2022 International Conference on Management of Data. 2091--2104."},{"key":"e_1_2_1_9_1","unstructured":"Bellard. 2018. FFmpeg. https:\/\/ffmpeg.org. Accessed: 2023-05-11."},{"key":"e_1_2_1_10_1","volume-title":"2008 19th International Conference on Pattern Recognition. IEEE, 1--4.","author":"Benezeth Yannick","year":"2008","unstructured":"Yannick Benezeth, Pierre-Marc Jodoin, Bruno Emile, H\u00e9l\u00e8ne Laurent, and Christophe Rosenberger. 2008. Review and evaluation of commonly-implemented background subtraction algorithms. In 2008 19th International Conference on Pattern Recognition. IEEE, 1--4."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2016.7533003"},{"key":"e_1_2_1_12_1","volume-title":"Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934","author":"Bochkovskiy Alexey","year":"2020","unstructured":"Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. 2020. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020)."},{"key":"e_1_2_1_13_1","volume-title":"Fida El Baf, and Bertrand Vachon","author":"Bouwmans Thierry","year":"2008","unstructured":"Thierry Bouwmans, Fida El Baf, and Bertrand Vachon. 2008. Background modeling using mixture of gaussians for foreground detection-a survey. Recent patents on computer science 1, 3 (2008), 219--237."},{"key":"e_1_2_1_14_1","volume-title":"Overview of the versatile video coding (VVC) standard and its applications","author":"Bross Benjamin","year":"2021","unstructured":"Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J Sullivan, and Jens-Rainer Ohm. 2021. Overview of the versatile video coding (VVC) standard and its applications. IEEE Transactions on Circuits and Systems for Video Technology (2021)."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3517857"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE51399.2021.00156"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3459242"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.14778\/3231751.3231768"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_21_1","volume-title":"A texture-based method for modeling the background and detecting moving objects","author":"Heikkila Marko","year":"2006","unstructured":"Marko Heikkila and Matti Pietikainen. 2006. A texture-based method for modeling the background and detecting moving objects. IEEE transactions on pattern analysis and machine intelligence 28, 4 (2006), 657--662."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/SEC.2018.00016"},{"key":"e_1_2_1_23_1","unstructured":"Glenn Jocher Ayush Chaurasia Alex Stoken Jirka Borovec Yonghye Kwon Kalen Michael Jiacong Fang Zeng Yifu Colin Wong Diego Montes et al. 2022. ultralytics\/yolov5: v7. 0-YOLOv5 SOTA Realtime Instance Segmentation. Zenodo (2022)."},{"key":"e_1_2_1_24_1","volume-title":"BlazeIt: optimizing declarative aggregation and limit queries for neural network-based video analytics. arXiv preprint arXiv:1805.01046","author":"Kang Daniel","year":"2018","unstructured":"Daniel Kang, Peter Bailis, and Matei Zaharia. 2018. BlazeIt: optimizing declarative aggregation and limit queries for neural network-based video analytics. arXiv preprint arXiv:1805.01046 (2018)."},{"key":"e_1_2_1_25_1","volume-title":"Noscope: optimizing neural network queries over video at scale. arXiv preprint arXiv:1703.02529","author":"Kang Daniel","year":"2017","unstructured":"Daniel Kang, John Emmons, Firas Abuzaid, Peter Bailis, and Matei Zaharia. 2017. Noscope: optimizing neural network queries over video at scale. arXiv preprint arXiv:1703.02529 (2017)."},{"key":"e_1_2_1_26_1","volume-title":"Approximate selection with guarantees using proxies. arXiv preprint arXiv:2004.00827","author":"Kang Daniel","year":"2020","unstructured":"Daniel Kang, Edward Gan, Peter Bailis, Tatsunori Hashimoto, and Matei Zaharia. 2020. Approximate selection with guarantees using proxies. arXiv preprint arXiv:2004.00827 (2020)."},{"key":"e_1_2_1_27_1","volume-title":"Semantic Indexes for Machine Learning-based Queries over Unstructured Data. arXiv e-prints","author":"Kang Daniel","year":"2020","unstructured":"Daniel Kang, John Guibas, Peter Bailis, Tatsunori Hashimoto, and Matei Zaharia. 2020. Semantic Indexes for Machine Learning-based Queries over Unstructured Data. arXiv e-prints (2020), arXiv-2009."},{"key":"e_1_2_1_28_1","volume-title":"Jointly optimizing preprocessing and inference for DNN-based visual analytics. arXiv preprint arXiv:2007.13005","author":"Kang Daniel","year":"2020","unstructured":"Daniel Kang, Ankit Mathur, Teja Veeramacheneni, Peter Bailis, and Matei Zaharia. 2020. Jointly optimizing preprocessing and inference for DNN-based visual analytics. arXiv preprint arXiv:2007.13005 (2020)."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2833157.2833162"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3387514.3405874"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.324"},{"key":"e_1_2_1_32_1","first-page":"1390","article-title":"Fast Human Pose Estimation in Compressed Videos","volume":"25","author":"Liu Huan","year":"2022","unstructured":"Huan Liu, Wentao Liu, Zhixiang Chi, Yang Wang, Yuanhao Yu, Jun Chen, and Jin Tang. 2022. Fast Human Pose Estimation in Compressed Videos. IEEE Transactions on Multimedia 25 (2022), 1390--1400.","journal-title":"IEEE Transactions on Multimedia"},{"key":"e_1_2_1_33_1","volume-title":"Real-time online multi-object tracking in compressed domain. arXiv preprint arXiv:2204.02081","author":"Liu Qiankun","year":"2022","unstructured":"Qiankun Liu, Bin Liu, Yue Wu, Weihai Li, and Nenghai Yu. 2022. Real-time online multi-object tracking in compressed domain. arXiv preprint arXiv:2204.02081 (2022)."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1014614221234"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2987550.2987564"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.5555\/1222671.1222999"},{"key":"e_1_2_1_37_1","volume-title":"2022 IEEE 38th International Conference on Data Engineering (ICDE). IEEE, 2956--2968","author":"Moll Oscar","year":"2022","unstructured":"Oscar Moll, Favyen Bastani, Sam Madden, Mike Stonebraker, Vijay Gadepally, and Tim Kraska. 2022. Exsample: Efficient searches on video repositories through adaptive sampling. In 2022 IEEE 38th International Conference on Data Engineering (ICDE). IEEE, 2956--2968."},{"key":"e_1_2_1_38_1","unstructured":"Nvidia. 2013. Nvidia Video Codec. https:\/\/developer.nvidia.com\/video-codec-sdk. Accessed: 2023-05-11."},{"key":"e_1_2_1_39_1","unstructured":"Nvidia. 2019. Video Processing Framework. https:\/\/github.com\/NVIDIA\/VideoProcessingFramework. Accessed: 2023-05-11."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201394"},{"key":"e_1_2_1_41_1","volume-title":"Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767","author":"Redmon Joseph","year":"2018","unstructured":"Joseph Redmon and Ali Farhadi. 2018. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 (2018)."},{"key":"e_1_2_1_42_1","unstructured":"Shaoqing Ren Kaiming He Ross Girshick and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems. 91--99."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.14778\/3570690.3570695"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2007.905532"},{"key":"e_1_2_1_45_1","first-page":"9","article-title":"Moving object detection using statistical background subtraction in wavelet compressed domain","volume":"79","author":"Sengar Sandeep Singh","year":"2020","unstructured":"Sandeep Singh Sengar and Susanta Mukhopadhyay. 2020. Moving object detection using statistical background subtraction in wavelet compressed domain. Multimedia Tools and Applications 79, 9-10 (2020), 5919--5940.","journal-title":"Multimedia Tools and Applications"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.895972"},{"key":"e_1_2_1_47_1","volume-title":"Improving background subtraction using local binary similarity patterns","author":"St-Charles Pierre-Luc","unstructured":"Pierre-Luc St-Charles and Guillaume-Alexandre Bilodeau. 2014. Improving background subtraction using local binary similarity patterns. In IEEE winter conference on applications of computer vision. IEEE, 509--515."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2012.2221191"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00972"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/SEC.2018.00019"},{"key":"e_1_2_1_51_1","volume-title":"Proceedings of the 3rd Workshop on Hot Topics in Wireless. 2--6.","author":"Wang Xiaoli","year":"2016","unstructured":"Xiaoli Wang, Aakanksha Chowdhery, and Mung Chiang. 2016. SkyEyes: adaptive video streaming from UAVs. In Proceedings of the 3rd Workshop on Hot Topics in Wireless. 2--6."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2017.8296962"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.634"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3302424.3303971"},{"key":"e_1_2_1_55_1","volume-title":"2007 IEEE conference on computer vision and pattern recognition. IEEE, 1--8.","author":"Yao Jian","year":"2007","unstructured":"Jian Yao and Jean-Marc Odobez. 2007. Multi-layer background subtraction based on color and texture. In 2007 IEEE conference on computer vision and pattern recognition. IEEE, 1--8."},{"key":"e_1_2_1_56_1","volume-title":"14th USENIX Symposium on Networked Systems Design and Implementation.","author":"Zhang Haoyu","year":"2017","unstructured":"Haoyu Zhang, Ganesh Ananthanarayanan, Peter Bodik, Matthai Philipose, Paramvir Bahl, and Michael J Freedman. 2017. Live video analytics at scale with approximation and delay-tolerance. In 14th USENIX Symposium on Networked Systems Design and Implementation."},{"key":"e_1_2_1_57_1","volume-title":"Resnest: Split-attention networks. arXiv preprint arXiv:2004.08955","author":"Zhang Hang","year":"2020","unstructured":"Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Haibin Lin, Zhi Zhang, Yue Sun, Tong He, Jonas Mueller, R Manmatha, et al. 2020. Resnest: Split-attention networks. arXiv preprint arXiv:2004.08955 (2020)."},{"key":"e_1_2_1_58_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2490823","article-title":"Image search---from thousands to billions in 20 years","volume":"9","author":"Zhang Lei","year":"2013","unstructured":"Lei Zhang and Yong Rui. 2013. Image search---from thousands to billions in 20 years. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 9, 1s (2013), 1--20.","journal-title":"ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM)"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR.2004.1333992"},{"key":"e_1_2_1_60_1","volume-title":"Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern recognition letters 27, 7","author":"Zivkovic Zoran","year":"2006","unstructured":"Zoran Zivkovic and Ferdinand Van Der Heijden. 2006. Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern recognition letters 27, 7 (2006), 773--780."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3636218.3636224","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,5]],"date-time":"2024-03-05T12:07:01Z","timestamp":1709640421000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3636218.3636224"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12]]},"references-count":60,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["10.14778\/3636218.3636224"],"URL":"https:\/\/doi.org\/10.14778\/3636218.3636224","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2023,12]]},"assertion":[{"value":"2024-03-05","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}