{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T16:40:31Z","timestamp":1773247231389,"version":"3.50.1"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2024,5,16]],"date-time":"2024-05-16T00:00:00Z","timestamp":1715817600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61976029"],"award-info":[{"award-number":["61976029"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Key Project of Chongqing Technology Innovation and Application Development","award":["cstc2021jscx-gksbX0033"],"award-info":[{"award-number":["cstc2021jscx-gksbX0033"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2024,7,31]]},"abstract":"<jats:p>Once a video sequence is organized as basic shot units, it is of great interest to temporally link shots into semantic-compact scene segments to facilitate long video understanding. However, it still challenges existing video scene boundary detection methods to handle various visual semantics and complex shot relations in video scenes. We proposed a novel self-supervised learning method, Video Scene Montage for Boundary Detection (VSMBD), to extract rich shot semantics and learn shot relations using unlabeled videos. More specifically, we present Video Scene Montage (VSM) to synthesize reliable pseudo scene boundaries, which learns task-related semantic relations between shots in a self-supervised manner. To lay a solid foundation for modeling semantic relations between shots, we decouple visual semantics of shots into foreground and background. Instead of costly learning from scratch as in most previous self-supervised learning methods, we build our model upon large-scale pre-trained visual encoders to extract the foreground and background features. Experimental results demonstrate VSMBD trains a model with strong capability in capturing shot relations, surpassing previous methods by significant margins. The code is available at https:\/\/github.com\/mini-mind\/VSMBD.<\/jats:p>","DOI":"10.1145\/3654669","type":"journal-article","created":{"date-parts":[[2024,3,26]],"date-time":"2024-03-26T12:14:14Z","timestamp":1711455254000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Temporal Scene Montage for Self-Supervised Video Scene Boundary Detection"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3299-0785","authenticated-orcid":false,"given":"Jiawei","family":"Tan","sequence":"first","affiliation":[{"name":"Key Laboratory of Dependable Service Computing in Cyber Physical Society (Chongqing University), Ministry of Education, China, and School of Big Data and Software Engineering, Chongqing University,, Chongqing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9797-3241","authenticated-orcid":false,"given":"Pingan","family":"Yang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Dependable Service Computing in Cyber Physical Society (Chongqing University), Ministry of Education, China, and School of Big Data and Software Engineering, Chongqing University, Chongqing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-7297-2071","authenticated-orcid":false,"given":"Lu","family":"Chen","sequence":"additional","affiliation":[{"name":"Key Laboratory ofDependable Service Computing in Cyber Physical Society (Chongqing University), Ministry of Education, China, and School of Big Data and Software Engineering, Chongqing University, Chongqing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7799-1023","authenticated-orcid":false,"given":"Hongxing","family":"Wang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Dependable Service Computing in Cyber Physical Society (Chongqing University), Ministry of Education, China, and School of Big Data and Software Engineering, Chongqing University, Chongqing China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,5,16]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"179","volume-title":"WACV","author":"Ahsan Unaiza","year":"2019","unstructured":"Unaiza Ahsan, Rishi Madhok, and Irfan A. Essa. 2019. Video Jigsaw: Unsupervised learning of spatiotemporal context for video action recognition. In WACV. 179\u2013189."},{"key":"e_1_3_1_3_2","first-page":"1199","volume-title":"ACM Multimedia","author":"Baraldi Lorenzo","year":"2015","unstructured":"Lorenzo Baraldi, Costantino Grana, and Rita Cucchiara. 2015. A deep siamese network for scene detection in broadcast videos. In ACM Multimedia. 1199\u20131202."},{"key":"e_1_3_1_4_2","article-title":"Planet earth","year":"2006","unstructured":"BBC. 2006. Planet earth. Retrieved April 2, 2024 from https:\/\/www.bbc.co.uk\/programmes\/b006mywy","journal-title":"R"},{"key":"e_1_3_1_5_2","first-page":"9919","volume-title":"CVPR","author":"Benaim Sagie","year":"2020","unstructured":"Sagie Benaim, Ariel Ephrat, Oran Lang, Inbar Mosseri, William T. Freeman, Michael Rubinstein, Michal Irani, and Tali Dekel. 2020. SpeedNet: Learning the speediness in videos. In CVPR. 9919\u20139928."},{"key":"e_1_3_1_6_2","unstructured":"Brandon Castellanos. 2014. PySceneDetect. Retrieved April 2 2024 from https:\/\/www.scenedetect.com"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2008.2008924"},{"key":"e_1_3_1_8_2","first-page":"737","volume-title":"ICIP","author":"Chen Lei","year":"2002","unstructured":"Lei Chen and M. Tamer \u00d6zsu. 2002. Rule-based scene extraction from video. In ICIP. 737\u2013740."},{"key":"e_1_3_1_9_2","first-page":"14","volume-title":"MMM","author":"Chen Lu","year":"2024","unstructured":"Lu Chen, Jiawei Tan, Pingan Yang, and Hongxing Wang. 2024. Semantic transition detection for self-supervised video scene segmentation. In MMM. 14\u201327."},{"key":"e_1_3_1_10_2","first-page":"6535","volume-title":"CVPR","author":"Chen Shixing","year":"2023","unstructured":"Shixing Chen, Chun-Hao Liu, Xiang Hao, Xiaohan Nie, Maxim Arap, and Raffay Hamid. 2023. Movies2Scenes: Using movie metadata to learn scene representation. In CVPR. 6535\u20136544."},{"key":"e_1_3_1_11_2","first-page":"9796","volume-title":"CVPR","author":"Chen Shixing","year":"2021","unstructured":"Shixing Chen, Xiaohan Nie, David Fan, Dongqing Zhang, Vimal Bhat, and Raffay Hamid. 2021. Shot contrastive self-supervised learning for scene boundary detection. In CVPR. 9796\u20139805."},{"issue":"1","key":"e_1_3_1_12_2","first-page":"12:1\u201312:22","article-title":"Weakly supervised text-based actor-action video segmentation by clip-level multi-instance learning","volume":"19","author":"Chen Weidong","year":"2023","unstructured":"Weidong Chen, Guorong Li, Xinfeng Zhang, Shuhui Wang, Liang Li, and Qingming Huang. 2023. Weakly supervised text-based actor-action video segmentation by clip-level multi-instance learning. ACM Trans. Multim. Comput. Commun. Appl. 19, 1 (2023), 12:1\u201312:22.","journal-title":"ACM Trans. Multim. Comput. Commun. Appl."},{"key":"e_1_3_1_13_2","unstructured":"Joon Son Chung. 2019. Naver at ActivityNet Challenge 2019 - task B active speaker detection (AVA). arXiv:1906.10555. Retrieved April 2 2024 from https:\/\/arxiv.org\/abs\/1906.10555"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_1_15_2","unstructured":"Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly Jakob Uszkoreit and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. 2021. In ICLR. 21 pages."},{"key":"e_1_3_1_16_2","volume-title":"NeurIPS","author":"Feichtenhofer Christoph","year":"2022","unstructured":"Christoph Feichtenhofer, Haoqi Fan, Yanghao Li, and Kaiming He. 2022. Masked autoencoders as spatiotemporal learners. In NeurIPS."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.607"},{"key":"e_1_3_1_18_2","first-page":"3198","volume-title":"ICCVW","author":"Gaikwad Bhagyashree","year":"2021","unstructured":"Bhagyashree Gaikwad, Ankita Sontakke, Manasi S. Patwardhan, Niranjan Pedanekar, and Shirish Subhash Karande. 2021. Plots to previews: Towards automatic movie preview retrieval using publicly available meta-data. In ICCVW. 3198\u20133207."},{"key":"e_1_3_1_19_2","first-page":"1243","volume-title":"ICML","author":"Gehring Jonas","year":"2017","unstructured":"Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. 2017. Convolutional sequence to sequence learning. In ICML. 1243\u20131252."},{"key":"e_1_3_1_20_2","volume-title":"ICLR","author":"Gu Albert","year":"2022","unstructured":"Albert Gu, Karan Goel, and Christopher R\u00e9. 2022. Efficiently modeling long sequences with structured state spaces. In ICLR."},{"key":"e_1_3_1_21_2","first-page":"1","volume-title":"ICME","author":"Han Bo","year":"2011","unstructured":"Bo Han and Weiguo Wu. 2011. Video scene segmentation using a novel boundary evaluation criterion and dynamic programming. In ICME. 1\u20136."},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2021.04.016"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.123"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_25_2","first-page":"558","volume-title":"CVPR","author":"He Tong","year":"2019","unstructured":"Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, and Mu Li. 2019. Bag of tricks for image classification with convolutional neural networks. In CVPR. 558\u2013567."},{"key":"e_1_3_1_26_2","first-page":"709","volume-title":"ECCV","author":"Huang Qingqiu","year":"2020","unstructured":"Qingqiu Huang, Yu Xiong, Anyi Rao, Jiaze Wang, and Dahua Lin. 2020. MovieNet: A holistic dataset for movie understanding. In ECCV. 709\u2013727."},{"key":"e_1_3_1_27_2","first-page":"164","volume-title":"AIRS","author":"Inoue Masashi","year":"2018","unstructured":"Masashi Inoue and Ryu Yasuhara. 2018. Considering conversation scenes in movie summarization. In AIRS. 164\u2013170."},{"key":"e_1_3_1_28_2","first-page":"18749","volume-title":"CVPR","author":"Islam Md Mohaiminul","year":"2023","unstructured":"Md Mohaiminul Islam, Mahmudul Hasan, Kishan Shamsundar Athrey, Tony Braskich, and Gedas Bertasius. 2023. Efficient movie scene detection using state-space transformers. In CVPR. 18749\u201318758."},{"key":"e_1_3_1_29_2","first-page":"367","volume-title":"CVPR","author":"Kender John R.","year":"1998","unstructured":"John R. Kender and Boon-Lock Yeo. 1998. Video scene segmentation via continuous video coherence. In CVPR. 367\u2013373."},{"key":"e_1_3_1_30_2","first-page":"8545","volume-title":"AAAI","author":"Kim Dahun","year":"2019","unstructured":"Dahun Kim, Donghyeon Cho, and In So Kweon. 2019. Self-supervised video representation learning with space-time cubic puzzles. In AAAI. 8545\u20138552."},{"key":"e_1_3_1_31_2","volume-title":"ICLR","author":"Kingma Diederik P.","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR."},{"key":"e_1_3_1_32_2","first-page":"13947","volume-title":"CVPR","author":"Li Congcong","year":"2022","unstructured":"Congcong Li, Xinyao Wang, Longyin Wen, Dexiang Hong, Tiejian Luo, and Libo Zhang. 2022. End-to-end compressed video representation learning for generic event boundary detection. In CVPR. IEEE, 13947\u201313956."},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2020.3042476"},{"issue":"1","key":"e_1_3_1_34_2","first-page":"35:1\u201335:21","article-title":"Exploring relations in untrimmed videos for self-supervised learning","volume":"18","author":"Luo Dezhao","year":"2022","unstructured":"Dezhao Luo, Yu Zhou, Bo Fang, Yucan Zhou, Dayan Wu, and Weiping Wang. 2022. Exploring relations in untrimmed videos for self-supervised learning. ACM Trans. Multim. Comput. Commun. Appl. 18, 1s (2022), 35:1\u201335:21.","journal-title":"ACM Trans. Multim. Comput. Commun. Appl."},{"key":"e_1_3_1_35_2","first-page":"4027","volume-title":"ACCV","author":"Mun Jonghwan","year":"2022","unstructured":"Jonghwan Mun, Minchul Shin, Gunsoo Han, Sangho Lee, Seongsu Ha, Joonseok Lee, and Eun-Sol Kim. 2022. BaSSL: Boundary-aware self-supervised learning for video scene segmentation. In ACCV. 4027\u20134043."},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.80"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2004.841694"},{"key":"e_1_3_1_38_2","first-page":"6964","volume-title":"CVPR","author":"Qian Rui","year":"2021","unstructured":"Rui Qian, Tianjian Meng, Boqing Gong, Ming-Hsuan Yang, Huisheng Wang, Serge J. Belongie, and Yin Cui. 2021. Spatiotemporal contrastive video representation learning. In CVPR. 6964\u20136974."},{"key":"e_1_3_1_39_2","first-page":"10143","volume-title":"CVPR","author":"Rao Anyi","year":"2020","unstructured":"Anyi Rao, Linning Xu, Yu Xiong, Guodong Xu, Qingqiu Huang, Bolei Zhou, and Dahua Lin. 2020. A local-to-global approach to multi-modal movie scene segmentation. In CVPR. 10143\u201310152."},{"key":"e_1_3_1_40_2","first-page":"343","volume-title":"CVPR","author":"Rasheed Zeeshan","year":"2003","unstructured":"Zeeshan Rasheed and Mubarak Shah. 2003. Scene detection in hollywood movies and TV shows. In CVPR. 343\u2013350."},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2005.858392"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1142\/S1793351X17400086"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/s005300050138"},{"issue":"4","key":"e_1_3_1_44_2","first-page":"94:1\u201394:23","article-title":"Characters link shots: Character attention network for movie scene segmentation","volume":"20","author":"Tan Jiawei","year":"2024","unstructured":"Jiawei Tan, Hongxing Wang, and Junsong Yuan. 2024. Characters link shots: Character attention network for movie scene segmentation. ACM Trans. Multim. Comput. Commun. Appl. 20, 4 (2024), 94:1\u201394:23.","journal-title":"ACM Trans. Multim. Comput. Commun. Appl."},{"key":"e_1_3_1_45_2","unstructured":"Jing Tan Yuhong Wang Gangshan Wu and Limin Wang. 2023. Temporal perceiver: A general architecture for arbitrary boundary detection. IEEE Trans. Pattern Anal. Mach. Intell. 45 10 (2023) 12506\u201312520."},{"key":"e_1_3_1_46_2","first-page":"827","volume-title":"CVPR","author":"Tapaswi Makarand","year":"2014","unstructured":"Makarand Tapaswi, Martin B\u00e4uml, and Rainer Stiefelhagen. 2014. StoryGraphs: Visualizing character interactions as a timeline. In CVPR. 827\u2013834."},{"key":"e_1_3_1_47_2","article-title":"Tslearn, A machine learning toolkit for time series data","author":"Tavenard Romain","year":"2020","unstructured":"Romain Tavenard, Johann Faouzi, Gilles Vandewiele, Felix Divo, Guillaume Androz, Chester Holtz, Marie Payne, Roman Yurchak, Marc Ru\u00dfwurm, Kushal Kolar, and Eli Woods. 2020. Tslearn, A machine learning toolkit for time series data. J. Mach. Learn. Res. 21, 118 (2020), 1\u20136.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2002.808084"},{"key":"e_1_3_1_49_2","first-page":"5998","volume-title":"NeurIPS","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS. 5998\u20136008."},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00371-002-0184-9"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46484-8_2"},{"key":"e_1_3_1_52_2","first-page":"14001","volume-title":"CVPR","author":"Wu Haoqian","year":"2022","unstructured":"Haoqian Wu, Keyu Chen, Yanan Luo, Ruizhi Qiao, Bo Ren, Haozhe Liu, Weicheng Xie, and Linlin Shen. 2022. Scene consistency representation learning for video scene segmentation. In CVPR. 14001\u201314010."},{"key":"e_1_3_1_53_2","first-page":"10334","volume-title":"CVPR","author":"Xu Dejing","year":"2019","unstructured":"Dejing Xu, Jun Xiao, Zhou Zhao, Jian Shao, Di Xie, and Yueting Zhuang. 2019. Self-supervised spatiotemporal learning via video clip order prediction. In CVPR. 10334\u201310343."},{"key":"e_1_3_1_54_2","first-page":"7200","volume-title":"ICCV","author":"Xu Mengmeng","year":"2021","unstructured":"Mengmeng Xu, Juan-Manuel P\u00e9rez-R\u00faa, Victor Escorcia, Brais Mart\u00ednez, Xiatian Zhu, Li Zhang, Bernard Ghanem, and Tao Xiang. 2021. Boundary-sensitive pre-training for temporal localization in videos. In ICCV. 7200\u20137210."},{"key":"e_1_3_1_55_2","unstructured":"Haosen Yang Deng Huang Bin Wen Jiannan Wu Hongxun Yao Yi Jiang Xiatian Zhu and Zehuan Yuan. 2022. Self-supervised video representation learning with motion-aware masked autoencoders. arXiv:2210.04154. Retrieved April 2 2024 from https:\/\/arxiv.org\/abs\/2210.04154"},{"key":"e_1_3_1_56_2","first-page":"3206","volume-title":"AAAI","author":"Yang Yang","year":"2023","unstructured":"Yang Yang, Yurui Huang, Weili Guo, Baohua Xu, and Dingyin Xia. 2023. Towards global video scene segmentation with context-aware transformer. In AAAI. 3206\u20133213."},{"key":"e_1_3_1_57_2","first-page":"6547","volume-title":"CVPR","author":"Yao Yuan","year":"2020","unstructured":"Yuan Yao, Chang Liu, Dezhao Luo, Yu Zhou, and Qixiang Ye. 2020. Video playback rate perception for self-supervised spatio-temporal representation learning. In CVPR. 6547\u20136556."},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1006\/cviu.1997.0628"},{"key":"e_1_3_1_59_2","doi-asserted-by":"crossref","unstructured":"Ye Yuan and Jiawan Zhang. 2023. Shot boundary detection using color clustering and attention mechanism. ACM Trans. Multim. Comput. Commun. Appl. 19 6 (2023) 198:1\u2013198:23.","DOI":"10.1145\/3595923"},{"key":"e_1_3_1_60_2","volume-title":"CVPR","author":"Zhao Yanjun","year":"2007","unstructured":"Yanjun Zhao, Tao Wang, Peng Wang, Wei Hu, Yangzhou Du, Yimin Zhang, and Guangyou Xu. 2007. Scene segmentation and categorization using NCuts. In CVPR."},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2723009"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654669","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3654669","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:06:10Z","timestamp":1750291570000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654669"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,16]]},"references-count":60,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2024,7,31]]}},"alternative-id":["10.1145\/3654669"],"URL":"https:\/\/doi.org\/10.1145\/3654669","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,16]]},"assertion":[{"value":"2023-07-20","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-20","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}