{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,4]],"date-time":"2026-07-04T16:41:24Z","timestamp":1783183284715,"version":"3.54.6"},"reference-count":33,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2014,6,1]],"date-time":"2014-06-01T00:00:00Z","timestamp":1401580800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2014,6]]},"abstract":"<jats:p>This article presents an overview of the video copy detection benchmark which was run over a period of 4 years (2008--2011) as part of the TREC Video Retrieval (TRECVID) workshop series. The main contributions of the article include i) an examination of the evolving design of the evaluation framework and its components (system tasks, data, measures); ii) a high-level overview of results and best-performing approaches; and iii) a discussion of lessons learned over the four years. The content-based copy detection (CCD) benchmark worked with a large collection of synthetic queries, which is atypical for TRECVID, as was the use of a normalized detection cost framework. These particular evaluation design choices are motivated and appraised.<\/jats:p>","DOI":"10.1145\/2629531","type":"journal-article","created":{"date-parts":[[2014,7,22]],"date-time":"2014-07-22T15:08:20Z","timestamp":1406041700000},"page":"1-40","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":27,"title":["Content-Based Video Copy Detection Benchmarking at TRECVID"],"prefix":"10.1145","volume":"32","author":[{"given":"George","family":"Awad","sequence":"first","affiliation":[{"name":"Dakota Consulting, Inc., Silver Spring, MD"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paul","family":"Over","sequence":"additional","affiliation":[{"name":"National Institute of Standards and Technology, Gaithersburg, MD"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wessel","family":"Kraaij","sequence":"additional","affiliation":[{"name":"TNO, Radboud University, Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2014,7,8]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'04)","author":"Albiol A.","unstructured":"A. Albiol , M. Full , A. Albiol , and L. Torres . 2004. Detection of TV commercials . In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'04) . 541--544. A. Albiol, M. Full, A. Albiol, and L. Torres. 2004. Detection of TV commercials. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'04). 541--544."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1816041.1816072"},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP'07)","author":"Baluja S.","unstructured":"S. Baluja and M. Covell . 2007. Audio Ffingerprinting: Combining computer vision & data stream processing . In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP'07) . II.213--II.216. S. Baluja and M. Covell. 2007. Audio Ffingerprinting: Combining computer vision & data stream processing. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP'07). II.213--II.216."},{"key":"e_1_2_1_5_1","volume-title":"YouTube or YouLose&quest","author":"Breen Jason C.","year":"1950","unstructured":"Jason C. Breen . 2007. YouTube or YouLose&quest ; Can YouTube Survive a Copyright Infringement Lawsuit. bepress Legal Series , 1950 . Jason C. Breen. 2007. YouTube or YouLose&quest; Can YouTube Survive a Copyright Infringement Lawsuit. bepress Legal Series, 1950."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR.2006.187"},{"key":"e_1_2_1_7_1","volume-title":"INRIA-IMEDIA TRECVID 2008: Video copy detection. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv8.papers\/inria-lear.pdf.","author":"Douze Matthijs","year":"2008","unstructured":"Matthijs Douze , Adrien Gaidon , Herve Jegou , Marcin Marszlak , and Cordelia Schmid . 2008 . INRIA-IMEDIA TRECVID 2008: Video copy detection. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv8.papers\/inria-lear.pdf. (2008). Matthijs Douze, Adrien Gaidon, Herve Jegou, Marcin Marszlak, and Cordelia Schmid. 2008. INRIA-IMEDIA TRECVID 2008: Video copy detection. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv8.papers\/inria-lear.pdf. (2008)."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2010.2046265"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the NEM Summit.","author":"Eendebak Pieter","year":"2008","unstructured":"Pieter Eendebak , Wessel Kraaij , Stephan Raaijmakers , Elena Ranguelova , Ork de Rooij , Andrew Thean , and Marcel Worring . 2008 . Visual tools for assisting child pornography investigators . In Proceedings of the NEM Summit. Pieter Eendebak, Wessel Kraaij, Stephan Raaijmakers, Elena Ranguelova, Ork de Rooij, Andrew Thean, and Marcel Worring. 2008. Visual tools for assisting child pornography investigators. In Proceedings of the NEM Summit."},{"key":"e_1_2_1_10_1","volume-title":"Doddington","author":"Fiscus Jonathan G.","year":"2002","unstructured":"Jonathan G. Fiscus and George R . Doddington . 2002 . Topic Detection and Tracking Evaluation Overview. Kluwer , Norwell, MA, 17--31. http:\/\/dl.acm.org\/citation.cfm&quest;id=772260.772263. Jonathan G. Fiscus and George R. Doddington. 2002. Topic Detection and Tracking Evaluation Overview. Kluwer, Norwell, MA, 17--31. http:\/\/dl.acm.org\/citation.cfm&quest;id=772260.772263."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-010-0608-x"},{"key":"e_1_2_1_12_1","volume-title":"CRIM AT TRECVID 2011: Content-based copy detection using nearest neighbor mapping. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv11","author":"Gupta Vishwa","year":"2011","unstructured":"Vishwa Gupta , Parisa Darvish Zadeh Varcheie , Langis Gagnon , and Gilles Boulianne . 2011 . CRIM AT TRECVID 2011: Content-based copy detection using nearest neighbor mapping. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv11 .papers\/crim.ccd.pdf. Vishwa Gupta, Parisa Darvish Zadeh Varcheie, Langis Gagnon, and Gilles Boulianne. 2011. CRIM AT TRECVID 2011: Content-based copy detection using nearest neighbor mapping. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv11.papers\/crim.ccd.pdf."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSPA.2012.6310685"},{"key":"e_1_2_1_14_1","unstructured":"Maguelonne H\u00e9ritier Vishwa Gupta Langis Gagnon Gilles Boulianne Samuel Foucher and Patrick Cardinal. 2009. CRIMs content-based copy detection system for TRECVID. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv9.papers\/crim.pdf. (2009).  Maguelonne H\u00e9ritier Vishwa Gupta Langis Gagnon Gilles Boulianne Samuel Foucher and Patrick Cardinal. 2009. CRIMs content-based copy detection system for TRECVID. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv9.papers\/crim.pdf. (2009)."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2010.2050737"},{"key":"e_1_2_1_17_1","unstructured":"Menglin Jian Shu Fang Yonghong Tian Tiejun Huang and Wen Gao. 2011. PKU-IDM&commat;TRECVid 2011 CBCD: Content-based copy detection with cascade of multimodal features and temporal pyramid matching. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv11.papers\/pku-ccd.pdf. (2011).  Menglin Jian Shu Fang Yonghong Tian Tiejun Huang and Wen Gao. 2011. PKU-IDM&commat;TRECVid 2011 CBCD: Content-based copy detection with cascade of multimodal features and temporal pyramid matching. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv11.papers\/pku-ccd.pdf. (2011)."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICME.2012.189"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2006.886278"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1027527.1027729"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1459359.1459406"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1002\/nav.3800020109"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1282280.1282336"},{"key":"e_1_2_1_24_1","doi-asserted-by":"crossref","unstructured":"J. Law-To A. Joly and N. Boujemaa. 2007b. Muscle-VCD-2007: A live benchmark for video copy detection. http:\/\/wwwrocq.inria.fr\/imedia\/civr-bench\/.  J. Law-To A. Joly and N. Boujemaa. 2007b. Muscle-VCD-2007: A live benchmark for video copy detection. http:\/\/wwwrocq.inria.fr\/imedia\/civr-bench\/.","DOI":"10.1145\/1282280.1282336"},{"key":"e_1_2_1_25_1","unstructured":"Yuanning Li Luntian Mou Menglin Jiang Chi Su Xiaoyu Fang Mengren Qian Yonghong Tian Yaowei Wang Tiejun Huang and Wen Gao. 2010. PKU-INM&commat; TRECVid 2010: Copy detection with visual-audio feature fusion and sequential pyramid matching. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv10.papers\/pku-idm-ccd.pdf. (2010).  Yuanning Li Luntian Mou Menglin Jiang Chi Su Xiaoyu Fang Mengren Qian Yonghong Tian Yaowei Wang Tiejun Huang and Wen Gao. 2010. PKU-INM&commat; TRECVid 2010: Copy detection with visual-audio feature fusion and sequential pyramid matching. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv10.papers\/pku-idm-ccd.pdf. (2010)."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2501654.2501658"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1743384.1743409"},{"key":"e_1_2_1_28_1","unstructured":"Zhu Liu Tao Liu and Behzad Shahraray. 2009. AT&T research at TRECVID 2009 content-based copy detection. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv9.papers\/att.pdf. (2009).  Zhu Liu Tao Liu and Behzad Shahraray. 2009. AT&T research at TRECVID 2009 content-based copy detection. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv9.papers\/att.pdf. (2009)."},{"key":"e_1_2_1_29_1","volume-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.","volume":"2","author":"Meng Yan","year":"2003","unstructured":"Yan Meng , E. Chang , and Beitao Li . 2003 . Enhancing DPF for near-replica image recognition . In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 2 . II--416--23. DOI:http:\/\/dx.doi.org\/10.1109\/CVPR.2003.1211498 10.1109\/CVPR.2003.1211498 Yan Meng, E. Chang, and Beitao Li. 2003. Enhancing DPF for near-replica image recognition. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 2. II--416--23. DOI:http:\/\/dx.doi.org\/10.1109\/CVPR.2003.1211498"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0031-3203(01)00251-5"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2011.6116504"},{"key":"e_1_2_1_32_1","volume-title":"Proceedings of the IEEE Conference on Cybernetics and Intelligent Systems. 738--743","author":"Wan Y. H.","year":"2008","unstructured":"Y. H. Wan , Q. L. Yuan , S. M. Ji , L. M. He , and Y. L. Wang . 2008. A survey of the image copy detection . In Proceedings of the IEEE Conference on Cybernetics and Intelligent Systems. 738--743 . DOI:http:\/\/dx.doi.org\/10.1109\/ICCIS. 2008 .4670942 10.1109\/ICCIS.2008.4670942 Y. H. Wan, Q. L. Yuan, S. M. Ji, L. M. He, and Y. L. Wang. 2008. A survey of the image copy detection. In Proceedings of the IEEE Conference on Cybernetics and Intelligent Systems. 738--743. DOI:http:\/\/dx.doi.org\/10.1109\/ICCIS.2008.4670942"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1291233.1291280"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-35740-4_26"},{"key":"e_1_2_1_35_1","volume-title":"TRECVID 2009 of MCG-ICT-CAS. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv9.papers\/mcg-ict-cas.pdf.","author":"Zhang Yong-Dong","year":"2009","unstructured":"Yong-Dong Zhang , Ke Gao , Xiao Wu , Hon-Ttao Xie , Wei Zhang , and Zhen-Dong Mao . 2009 . TRECVID 2009 of MCG-ICT-CAS. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv9.papers\/mcg-ict-cas.pdf. (2009). Yong-Dong Zhang, Ke Gao, Xiao Wu, Hon-Ttao Xie, Wei Zhang, and Zhen-Dong Mao. 2009. TRECVID 2009 of MCG-ICT-CAS. http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv9.papers\/mcg-ict-cas.pdf. (2009)."}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2629531","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2629531","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T07:01:18Z","timestamp":1750230078000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2629531"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,6]]},"references-count":33,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2014,6]]}},"alternative-id":["10.1145\/2629531"],"URL":"https:\/\/doi.org\/10.1145\/2629531","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,6]]},"assertion":[{"value":"2013-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-07-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}