{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T13:32:59Z","timestamp":1771680779765,"version":"3.50.1"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2024,12,13]],"date-time":"2024-12-13T00:00:00Z","timestamp":1734048000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2020YFB1708500"],"award-info":[{"award-number":["2020YFB1708500"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2024,12,31]]},"abstract":"<jats:p>\n            Temporal action detection, a critical task in video activity understanding, is typically divided into two stages: proposal generation and classification. However, most existing methods overlook the importance of information transfer among proposals during classification, often treating each proposal in isolation, which hampers accurate label prediction. In this article, we propose a novel method for inferring semantic relationships both within and between action proposals, guiding the fusion of action proposal features accordingly. Building on this approach, we introduce the Proposal Semantic Relationship Graph Network (PSRGN), an end-to-end model that leverages intra-proposal semantic relationship graphs to extract cross-scale temporal context and an inter-proposal semantic relationship graph to incorporate complementary neighboring information, significantly improving proposal feature quality and overall detection performance. This is the first method to apply graph structure learning in temporal action detection, adaptively constructing the inter-proposal semantic graph. Extensive experiments on two datasets demonstrate the effectiveness of our approach, achieving state-of-the-art (SOTA). Code and results are available at\n            <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"http:\/\/github.com\/Riiick2011\/PSRGN\">http:\/\/github.com\/Riiick2011\/PSRGN<\/jats:ext-link>\n            .\n          <\/jats:p>","DOI":"10.1145\/3702233","type":"journal-article","created":{"date-parts":[[2024,10,28]],"date-time":"2024-10-28T16:55:51Z","timestamp":1730134551000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Proposal Semantic Relationship Graph Network for Temporal Action Detection"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-5707-932X","authenticated-orcid":false,"given":"Shaowen","family":"Su","sequence":"first","affiliation":[{"name":"Beijing Institute of Technology, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9125-3630","authenticated-orcid":false,"given":"Yan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Beijing Institute of Technology, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2163-2475","authenticated-orcid":false,"given":"Minggang","family":"Gan","sequence":"additional","affiliation":[{"name":"Beijing Institute of Technology, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,12,13]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW54120.2021.00356"},{"key":"e_1_3_1_3_2","doi-asserted-by":"crossref","unstructured":"Anurag Bagchi Jazib Mahmood Dolton Fernandes and Ravi Kiran Sarvadevabhatla. 2021. Hear me out: Fusional approaches for audio augmented temporal action localization. arXiv:2106.14118. Retrieved from https:\/\/arxiv.org\/abs\/2106.14118","DOI":"10.5220\/0010832700003124"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298698"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.502"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2021.3104226"},{"key":"e_1_3_1_7_2","unstructured":"Krzysztof Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane Tamas Sarlos Peter Hawkins Jared Davis Afroz Mohiuddin Lukasz Kaiser et al. 2020. Rethinking attention with performers. arXiv:2009.14794. Retrieved from https:\/\/arxiv.org\/abs\/2009.14794"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240508.3240668"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3531230"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3474085.3475439"},{"key":"e_1_3_1_11_2","first-page":"1972","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Franceschi Luca","year":"2019","unstructured":"Luca Franceschi, Mathias Niepert, Massimiliano Pontil, and Xiao He. 2019. Learning discrete structures for graph neural networks. In Proceedings of the International Conference on Machine Learning, 1972\u20131982."},{"key":"e_1_3_1_12_2","doi-asserted-by":"crossref","unstructured":"Ming-Gang Gan and Yan Zhang. 2022. Temporal attention-pyramid pooling for temporal action detection. IEEE Transactions on Multimedia (2022).","DOI":"10.1109\/TMM.2022.3166025"},{"key":"e_1_3_1_13_2","first-page":"21","article-title":"Temporal-visual proposal graph network for temporal action detection","volume":"53","author":"Gan Ming-Gang","year":"2023","unstructured":"Ming-Gang Gan, Yan Zhang, and Shaowen Su. 2023. Temporal-visual proposal graph network for temporal action detection. Applied Intelligence 53, 21 (2023), 26008\u201326026.","journal-title":"Applied Intelligence"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01157"},{"key":"e_1_3_1_15_2","volume-title":"Proceedings of the IEEE International Conference on Computer Vision (ICCV)","author":"Jiang Yu-Gang","year":"2014","unstructured":"Yu-Gang Jiang, Jingen Liu, A. Roshan Zamir, George Toderici, Ivan Laptev, Mubarak Shah, and Rahul Sukthankar. 2014. THUMOS challenge: Action recognition with a large number of classes. In Proceedings of the IEEE International Conference on Computer Vision (ICCV)."},{"key":"e_1_3_1_16_2","unstructured":"Thomas N. Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv:1609.02907. Retrieved from https:\/\/arxiv.org\/abs\/1609.02907"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.390"},{"key":"e_1_3_1_18_2","first-page":"4626","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Li Jin","year":"2020","unstructured":"Jin Li, Xianglong Liu, Zhuofan Zong, Wanru Zhao, Mingyuan Zhang, and Jingkuan Song. 2020. Graph attention based proposal 3d convnets for action detection. In Proceedings of the AAAI Conference on Artificial Intelligence, 4626\u20134633."},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11691"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394171.3413860"},{"key":"e_1_3_1_21_2","first-page":"3320","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Lin Chuming","year":"2021","unstructured":"Chuming Lin, Chengming Xu, Donghao Luo, Yabiao Wang, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, and Yanwei Fu. 2021. Learning salient boundary feature for anchor-free temporal action localization. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 3320\u20133329."},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.324"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i07.6829"},{"key":"e_1_3_1_24_2","first-page":"20010","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Liu Xiaolong","year":"2022","unstructured":"Xiaolong Liu, Song Bai, and Xiang Bai. 2022. An empirical study of end-to-end temporal action detection. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 20010\u201320019."},{"key":"e_1_3_1_25_2","first-page":"12596","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Liu Xiaolong","year":"2021","unstructured":"Xiaolong Liu, Yao Hu, Song Bai, Fei Ding, Xiang Bai, and Philip HS Torr. 2021. Multi-shot temporal event localization: A benchmark. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 12596\u201312606."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2022.3195321"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00043"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01390"},{"key":"e_1_3_1_29_2","first-page":"10362","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Nag Sauradip","year":"2023","unstructured":"Sauradip Nag, Xiatian Zhu, Jiankang Deng, Yi-Zhe Song, and Tao Xiang. 2023. DiffTAD: Temporal action detection with proposal denoising diffusion. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, 10362\u201310374."},{"key":"e_1_3_1_30_2","first-page":"645","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Nag Sauradip","year":"2022","unstructured":"Sauradip Nag, Xiatian Zhu, Yi-Zhe Song, and Tao Xiang. 2022. Temporal action detection with global segmentation mask learning. In Proceedings of the European Conference on Computer Vision. Springer, 645\u2013662."},{"key":"e_1_3_1_31_2","first-page":"267","volume-title":"Proceedings of the International Conference on Multimedia Retrieval","author":"Ren Yifan","year":"2021","unstructured":"Yifan Ren, Xing Xu, Fumin Shen, Zheng Wang, Yang Yang, and Heng Tao Shen. 2021. Multi-scale dynamic network for temporal action detection. In Proceedings of the International Conference on Multimedia Retrieval, 267\u2013275."},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00075"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01230"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2017.58"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2020.3044218"},{"key":"e_1_3_1_36_2","first-page":"13526","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Tan Jing","year":"2021","unstructured":"Jing Tan, Jiaqi Tang, Limin Wang, and Gangshan Wu. 2021. Relaxed transformer decoders for direct action proposal generation. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, 13526\u201313535."},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00675"},{"key":"e_1_3_1_38_2","unstructured":"Petar Velivckovic Guillem Cucurull Arantxa Casanova Adriana Romero Pietro Lio and Yoshua Bengio. 2017. Graph attention networks. arXiv:1710.10903. Retrieved from https:\/\/arxiv.org\/abs\/1710.10903"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2021.3061289"},{"key":"e_1_3_1_40_2","unstructured":"Chenhao Wang Hongxiang Cai Yuxin Zou and Yichao Xiong. 2021. RGB stream is enough for temporal action detection. arXiv:2107.04362. Retrieved from https:\/\/arxiv.org\/abs\/2107.04362"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46484-8_2"},{"key":"e_1_3_1_42_2","first-page":"13566","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Wang Qiang","year":"2022","unstructured":"Qiang Wang, Yanhao Zhang, Yun Zheng, and Pan Pan. 2022. RCL: Recurrent continuous localization for temporal action detection. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 13566\u201313575."},{"key":"e_1_3_1_43_2","unstructured":"Jiannan Wu Peize Sun Shoufa Chen Jiewen Yang Zihao Qi Lan Ma and Ping Luo. 2021. Towards high-quality temporal action detection with sparse proposals. arXiv:2109.08847. Retrieved from https:\/\/arxiv.org\/abs\/2109.08847"},{"key":"e_1_3_1_44_2","first-page":"13884","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Xia Kun","year":"2022","unstructured":"Kun Xia, Le Wang, Sanping Zhou, Nanning Zheng, and Wei Tang. 2022. Learning to refactor action and co-occurrence features for temporal action localization. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 13884\u201313893."},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01017"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.12328"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2023.103692"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01264-9_42"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00719"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-19772-7_29"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2021.02.085"},{"key":"e_1_3_1_52_2","first-page":"13658","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Zhao Chen","year":"2021","unstructured":"Chen Zhao, Ali K. Thabet, and Bernard Ghanem. 2021. Video self-stitching graph network for temporal action localization. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, 13658\u201313667."},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.317"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3702233","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3702233","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:10:31Z","timestamp":1750295431000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3702233"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,13]]},"references-count":52,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,12,31]]}},"alternative-id":["10.1145\/3702233"],"URL":"https:\/\/doi.org\/10.1145\/3702233","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,13]]},"assertion":[{"value":"2023-11-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-12","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}