{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T12:23:07Z","timestamp":1775910187888,"version":"3.50.1"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2024,5,16]],"date-time":"2024-05-16T00:00:00Z","timestamp":1715817600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["LR22F020001 and LDT23F02025F02"],"award-info":[{"award-number":["LR22F020001 and LDT23F02025F02"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62125201, 62020106007 and 62072147"],"award-info":[{"award-number":["62125201, 62020106007 and 62072147"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2024,7,31]]},"abstract":"<jats:p>Video summarization remains a challenging task despite increasing research efforts. Traditional methods focus solely on long-range temporal modeling of video frames, overlooking important local motion information that cannot be captured by frame-level video representations. In this article, we propose the Parameter-free Motion Attention Module (PMAM) to exploit the crucial motion clues potentially contained in adjacent video frames, using a multi-head attention architecture. The PMAM requires no additional training for model parameters, leading to an efficient and effective understanding of video dynamics. Moreover, we introduce the Multi-feature Motion Attention Network (MMAN), integrating the PMAM with local and global multi-head attention based on object-centric and scene-centric video representations. The synergistic combination of local motion information, extracted by the proposed PMAM, with long-range interactions modeled by the local and global multi-head attention mechanism, can significantly enhance the performance of video summarization. Extensive experimental results on the benchmark datasets, SumMe and TVSum, demonstrate that the proposed MMAN outperforms other state-of-the-art methods, resulting in remarkable performance gains.<\/jats:p>","DOI":"10.1145\/3654670","type":"journal-article","created":{"date-parts":[[2024,3,30]],"date-time":"2024-03-30T09:24:47Z","timestamp":1711790687000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Effective Video Summarization by Extracting Parameter-Free Motion Attention"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2131-9200","authenticated-orcid":false,"given":"Tingting","family":"Han","sequence":"first","affiliation":[{"name":"College of Computer Science, Hangzhou Dianzi University, Hangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-0930-8688","authenticated-orcid":false,"given":"Quan","family":"Zhou","sequence":"additional","affiliation":[{"name":"Hangzhou Dianzi University, Hangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1922-7283","authenticated-orcid":false,"given":"Jun","family":"Yu","sequence":"additional","affiliation":[{"name":"Computer Science, Hangzhou Dianzi University, Hangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8407-1137","authenticated-orcid":false,"given":"Zhou","family":"Yu","sequence":"additional","affiliation":[{"name":"Hangzhou Dianzi University, Hangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0979-6514","authenticated-orcid":false,"given":"Jianhui","family":"Zhang","sequence":"additional","affiliation":[{"name":"Hangzhou Dianzi University, Hangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5843-6411","authenticated-orcid":false,"given":"Sicheng","family":"Zhao","sequence":"additional","affiliation":[{"name":"Tsinghua University Beijing National Research Center for Information Science and Technology, Beijing, China"}]}],"member":"320","published-online":{"date-parts":[[2024,5,16]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"226","volume-title":"Proceedings of the IEEE International Symposium on Multimedia","author":"Apostolidis Evlampios","year":"2021","unstructured":"Evlampios Apostolidis, Georgios Balaouras, Vasileios Mezaris, and Ioannis Patras. 2021. Combining global and local attention with positional encoding for video summarization. In Proceedings of the IEEE International Symposium on Multimedia. 226\u2013234."},{"key":"e_1_3_1_3_2","first-page":"184","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Cai Sijia","year":"2018","unstructured":"Sijia Cai, Wangmeng Zuo, Larry S. Davis, and Lei Zhang. 2018. Weakly-supervised video summarization using variational encoder-decoder and web prior. In Proceedings of the European Conference on Computer Vision. 184\u2013200."},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338533.3366583"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2010.08.004"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_1_7_2","first-page":"754","volume-title":"Proceedings of the IEEE Winter Conference on Applications of Computer Vision","author":"Elfeki Mohamed","year":"2019","unstructured":"Mohamed Elfeki and Ali Borji. 2019. Video summarization via actionness ranking. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision. 754\u2013763."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2511748"},{"key":"e_1_3_1_9_2","first-page":"39","volume-title":"Proceedings of the Asian Conference on Computer Vision","author":"Fajtl Jiri","year":"2018","unstructured":"Jiri Fajtl, Hajar Sadeghi Sokeh, Vasileios Argyriou, Dorothy Monekosso, and Paolo Remagnino. 2018. Summarizing videos with attention. In Proceedings of the Asian Conference on Computer Vision. 39\u201354."},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2020.12.016"},{"key":"e_1_3_1_11_2","first-page":"446","volume-title":"Proceedings of the International Conference on Pattern Recognition","author":"Fu Hao","year":"2021","unstructured":"Hao Fu, Hongxing Wang, and Jianyu Yang. 2021. Video summarization with a dual attention capsule network. In Proceedings of the International Conference on Pattern Recognition. IEEE, 446\u2013451."},{"key":"e_1_3_1_12_2","first-page":"1579","volume-title":"Proceedings of the IEEE Winter Conference on Applications of Computer Vision","author":"Fu Tsu-Jui","year":"2019","unstructured":"Tsu-Jui Fu, Shao-Heng Tai, and Hwann-Tzong Chen. 2019. Attentive and adversarial learning for video summarization. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision. IEEE, 1579\u20131587."},{"key":"e_1_3_1_13_2","article-title":"Vectorized evidential learning for weakly-supervised temporal action localization","author":"Gao Junyu","year":"2023","unstructured":"Junyu Gao, Mengyuan Chen, and Changsheng Xu. 2023. Vectorized evidential learning for weakly-supervised temporal action localization. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2020.3021980"},{"issue":"10","key":"e_1_3_1_15_2","first-page":"3476","article-title":"Learning to model relationships for zero-shot video classification","volume":"43","author":"Gao Junyu","year":"2020","unstructured":"Junyu Gao, Tianzhu Zhang, and Changsheng Xu. 2020. Learning to model relationships for zero-shot video classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 10 (2020), 3476\u20133491.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2632267"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10584-0_33"},{"key":"e_1_3_1_18_2","first-page":"1400","volume-title":"Proceedings of the ACM Symposium on Applied Computing","author":"Hadi Youssef","year":"2006","unstructured":"Youssef Hadi, Fedwa Essannouni, and Rachid Oulad Haj Thami. 2006. Video summarization by k-medoid clustering. In Proceedings of the ACM Symposium on Applied Computing. 1400\u20131401."},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2022.104532"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"issue":"2","key":"e_1_3_1_21_2","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1109\/TCSVT.2019.2890899","article-title":"A novel key-frames selection framework for comprehensive video summarization","volume":"30","author":"Huang Cheng","year":"2019","unstructured":"Cheng Huang and Hongmei Wang. 2019. A novel key-frames selection framework for comprehensive video summarization. IEEE Transactions on Circuits and Systems for Video Technology 30, 2 (2019), 577\u2013589.","journal-title":"IEEE Transactions on Circuits and Systems for Video Technology"},{"key":"e_1_3_1_22_2","article-title":"Conditional modeling based automatic video summarization","author":"Huang Jia-Hong","year":"2023","unstructured":"Jia-Hong Huang, Chao-Han Huck Yang, Pin-Yu Chen, Min-Hung Chen, and Marcel Worring. 2023. Conditional modeling based automatic video summarization. arXiv preprint arXiv:2311.12159 (2023).","journal-title":"arXiv preprint arXiv:2311.12159"},{"issue":"6","key":"e_1_3_1_23_2","first-page":"1709","article-title":"Video summarization with attention-based encoder-decoder networks","volume":"30","author":"Ji Zhong","year":"2019","unstructured":"Zhong Ji, Kailin Xiong, Yanwei Pang, and Xuelong Li. 2019. Video summarization with attention-based encoder-decoder networks. IEEE Transactions on Circuits and Systems for Video Technology 30, 6 (2019), 1709\u20131717.","journal-title":"IEEE Transactions on Circuits and Systems for Video Technology"},{"issue":"4","key":"e_1_3_1_24_2","first-page":"1765","article-title":"Deep attentive video summarization with distribution consistency learning","volume":"32","author":"Ji Zhong","year":"2020","unstructured":"Zhong Ji, Yuxiao Zhao, Yanwei Pang, Xi Li, and Jungong Han. 2020. Deep attentive video summarization with distribution consistency learning. IEEE Transactions on Neural Networks and Learning Systems 32, 4 (2020), 1765\u20131775.","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"e_1_3_1_25_2","first-page":"8537","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"33","author":"Jung Yunjae","year":"2019","unstructured":"Yunjae Jung, Donghyeon Cho, Dahun Kim, Sanghyun Woo, and In So Kweon. 2019. Discriminative feature learning for unsupervised video summarization. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 8537\u20138544."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460426.3463597"},{"key":"e_1_3_1_27_2","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1007\/978-3-030-05716-9_6","volume-title":"Proceedings of the International Conference on MultiMedia Modeling","author":"Casas Luis Lebron","year":"2019","unstructured":"Luis Lebron Casas and Eugenia Koblents. 2019. Video summarization with LSTM and deep attention models. In Proceedings of the International Conference on MultiMedia Modeling. 67\u201379."},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2020.107677"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485472"},{"key":"e_1_3_1_30_2","first-page":"3239","volume-title":"Proceedings of the IEEE Winter Conference on Applications of Computer Vision","author":"Li Zutong","year":"2021","unstructured":"Zutong Li and Lei Yang. 2021. Weakly supervised deep reinforcement learning for video summarization with semantically meaningful reward. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision. 3239\u20133247."},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2021.09.015"},{"key":"e_1_3_1_32_2","volume-title":"Proceedings of the Asian Conference on Computer Vision","author":"Liu Yen-Ting","year":"2020","unstructured":"Yen-Ting Liu, Yu-Jhe Li, and Yu-Chiang Frank Wang. 2020. Transforming multi-concept attention into video summarization. In Proceedings of the Asian Conference on Computer Vision."},{"key":"e_1_3_1_33_2","first-page":"3377","volume-title":"Proceedings of the IEEE International Conference on Image Processing","author":"Liu Yen-Ting","year":"2019","unstructured":"Yen-Ting Liu, Yu-Jhe Li, Fu-En Yang, Shang-Fu Chen, and Yu-Chiang Frank Wang. 2019. Learning hierarchical self-attention for video summarization. In Proceedings of the IEEE International Conference on Image Processing. 3377\u20133381."},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.318"},{"key":"e_1_3_1_35_2","doi-asserted-by":"crossref","first-page":"571","DOI":"10.23919\/EUSIPCO58844.2023.10289808","volume-title":"2023 31st European Signal Processing Conference (EUSIPCO)","author":"Minaidi Maria Nektaria","year":"2023","unstructured":"Maria Nektaria Minaidi, Charilaos Papaioannou, and Alexandros Potamianos. 2023. Self-attention based generative adversarial networks for unsupervised video summarization. In 2023 31st European Signal Processing Conference (EUSIPCO). IEEE, 571\u2013575."},{"key":"e_1_3_1_36_2","first-page":"13988","article-title":"CLIP-It! Language-guided video summarization","volume":"34","author":"Narasimhan Medhini","year":"2021","unstructured":"Medhini Narasimhan, Anna Rohrbach, and Trevor Darrell. 2021. CLIP-It! Language-guided video summarization. Advances in Neural Information Processing Systems 34 (2021), 13988\u201314000.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.395"},{"key":"e_1_3_1_38_2","first-page":"647","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Park Jungin","year":"2020","unstructured":"Jungin Park, Jiyoung Lee, Ig-Jae Kim, and Kwanghoon Sohn. 2020. SumGraph: Video summarization via recursive graph modeling. In Proceedings of the European Conference on Computer Vision. Springer, 647\u2013663."},{"key":"e_1_3_1_39_2","first-page":"347","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Rochan Mrigank","year":"2018","unstructured":"Mrigank Rochan, Linwei Ye, and Yang Wang. 2018. Video summarization using fully convolutional sequence networks. In Proceedings of the European Conference on Computer Vision. 347\u2013363."},{"key":"e_1_3_1_40_2","first-page":"5179","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Song Yale","year":"2015","unstructured":"Yale Song, Jordi Vallmitjana, Amanda Stent, and Alejandro Jaimes. 2015. TVSum: Summarizing web videos using titles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5179\u20135187."},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3571735"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00773"},{"key":"e_1_3_1_44_2","first-page":"4023","volume-title":"Proceedings of the ACM International Conference on Multimedia","author":"Wang Junyan","year":"2020","unstructured":"Junyan Wang, Yang Bai, Yang Long, Bingzhang Hu, Zhenhua Chai, Yu Guan, and Xiaolin Wei. 2020. Query twice: Dual mixture attention meta learning for video summarization. In Proceedings of the ACM International Conference on Multimedia. 4023\u20134031."},{"key":"e_1_3_1_45_2","article-title":"ERA: Entity relationship aware video summarization with Wasserstein GAN","author":"Wu Guande","year":"2021","unstructured":"Guande Wu, Jianzhe Lin, and Cl\u00e1udio T. Silva. 2021. ERA: Entity relationship aware video summarization with Wasserstein GAN. arXiv preprint arXiv:2109.02625 (2021).","journal-title":"arXiv preprint arXiv:2109.02625"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.112"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2021.03.026"},{"key":"e_1_3_1_48_2","first-page":"23191","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Yu Zhou","year":"2023","unstructured":"Zhou Yu, Lixiang Zheng, Zhou Zhao, Fei Wu, Jianping Fan, Kui Ren, and Jun Yu. 2023. ANetQA: A large-scale benchmark for fine-grained compositional reasoning over untrimmed videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 23191\u201323200."},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2019.2959451"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2916989"},{"key":"e_1_3_1_51_2","first-page":"5682","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Zhang Guozhen","year":"2023","unstructured":"Guozhen Zhang, Yuhan Zhu, Haonan Wang, Youxin Chen, Gangshan Wu, and Limin Wang. 2023. Extracting motion and appearance via inter-frame attention for efficient video frame interpolation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5682\u20135692."},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46478-7_47"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01237-3_24"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2021.10.039"},{"issue":"5","key":"e_1_3_1_55_2","first-page":"2793","article-title":"Reconstructive sequence-graph network for video summarization","volume":"44","author":"Zhao Bin","year":"2021","unstructured":"Bin Zhao, Haopeng Li, Xiaoqiang Lu, and Xuelong Li. 2021. Reconstructive sequence-graph network for video summarization. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 5 (2021), 2793\u20132801.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_1_56_2","doi-asserted-by":"crossref","first-page":"863","DOI":"10.1145\/3123266.3123328","volume-title":"Proceedings of the ACM International Conference on Multimedia","author":"Zhao Bin","year":"2017","unstructured":"Bin Zhao, Xuelong Li, and Xiaoqiang Lu. 2017. Hierarchical recurrent neural network for video summarization. In Proceedings of the ACM International Conference on Multimedia. 863\u2013871."},{"issue":"4","key":"e_1_3_1_57_2","doi-asserted-by":"crossref","first-page":"3629","DOI":"10.1109\/TIE.2020.2979573","article-title":"TTH-RNN: Tensor-train hierarchical recurrent neural network for video summarization","volume":"68","author":"Zhao Bin","year":"2020","unstructured":"Bin Zhao, Xuelong Li, and Xiaoqiang Lu. 2020. TTH-RNN: Tensor-train hierarchical recurrent neural network for video summarization. IEEE Transactions on Industrial Electronics 68, 4 (2020), 3629\u20133637.","journal-title":"IEEE Transactions on Industrial Electronics"},{"key":"e_1_3_1_58_2","first-page":"2513","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Zhao Bin","year":"2014","unstructured":"Bin Zhao and Eric P. Xing. 2014. Quasi real-time summarization for consumer videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2513\u20132520."},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477538"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2723009"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.12255"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2022.3163855"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2021.108312"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2020.3039886"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491228"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654670","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3654670","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:49:04Z","timestamp":1750286944000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654670"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,16]]},"references-count":64,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2024,7,31]]}},"alternative-id":["10.1145\/3654670"],"URL":"https:\/\/doi.org\/10.1145\/3654670","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,16]]},"assertion":[{"value":"2023-09-18","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-20","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}