{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T07:48:56Z","timestamp":1782373736421,"version":"3.54.5"},"reference-count":49,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2022,11,1]],"date-time":"2022-11-01T00:00:00Z","timestamp":1667260800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Sichuan Sciences and Technology Program","award":["2019ZDZX0005"],"award-info":[{"award-number":["2019ZDZX0005"]}]},{"name":"Sichuan Sciences and Technology Program","award":["2022ZHCG0007"],"award-info":[{"award-number":["2022ZHCG0007"]}]},{"name":"Sichuan Sciences and Technology Program","award":["2019ZDZX0005"],"award-info":[{"award-number":["2019ZDZX0005"]}]},{"name":"Sichuan Sciences and Technology Program","award":["2022ZHCG0007"],"award-info":[{"award-number":["2022ZHCG0007"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>As an important part of video understanding, temporal action detection (TAD) has wide application scenarios. It aims to simultaneously predict the boundary position and class label of every action instance in an untrimmed video. Most of the existing temporal action detection methods adopt a stacked convolutional block strategy to model long temporal structures. However, most of the information between adjacent frames is redundant, and distant information is weakened after multiple convolution operations. In addition, the durations of action instances vary widely, making it difficult for single-scale modeling to fit complex video structures. To address this issue, we propose a non-local temporal difference network (NTD), including a chunk convolution (CC) module, a multiple temporal coordination (MTC) module, and a temporal difference (TD) module. The TD module adaptively enhances the motion information and boundary features with temporal attention weights. The CC module evenly divides the input sequence into N chunks, using multiple independent convolution blocks to simultaneously extract features from neighboring chunks. Therefore, it realizes the information delivered from distant frames while avoiding trapping into the local convolution. The MTC module designs a cascade residual architecture, which realizes the multiscale temporal feature aggregation without introducing additional parameters. The NTD achieves a state-of-the-art performance on two large-scale datasets, 36.2% mAP@avg and 71.6% mAP@0.5 on ActivityNet-v1.3 and THUMOS-14, respectively.<\/jats:p>","DOI":"10.3390\/s22218396","type":"journal-article","created":{"date-parts":[[2022,11,2]],"date-time":"2022-11-02T03:36:44Z","timestamp":1667360204000},"page":"8396","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Non-Local Temporal Difference Network for Temporal Action Detection"],"prefix":"10.3390","volume":"22","author":[{"given":"Yilong","family":"He","sequence":"first","affiliation":[{"name":"Chengdu Institute of Computer Application, Chinese Academy of Sciences, Chengdu 610081, China"},{"name":"School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4488-5993","authenticated-orcid":false,"given":"Xiao","family":"Han","sequence":"additional","affiliation":[{"name":"Chengdu Institute of Computer Application, Chinese Academy of Sciences, Chengdu 610081, China"},{"name":"School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yong","family":"Zhong","sequence":"additional","affiliation":[{"name":"Chengdu Institute of Computer Application, Chinese Academy of Sciences, Chengdu 610081, China"},{"name":"School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lishun","family":"Wang","sequence":"additional","affiliation":[{"name":"Chengdu Institute of Computer Application, Chinese Academy of Sciences, Chengdu 610081, China"},{"name":"School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Wang, Q., Zhang, Y., Zheng, Y., and Pan, P. (2022, January 19\u201320). RCL: Recurrent Continuous Localization for Temporal Action Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01320"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Dai, R., Das, S., Minciullo, L., Garattoni, L., Francesca, G., and Bremond, F. (2021, January 19\u201325). Pdan: Pyramid dilated attention network for action detection. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Nashville, TN, USA.","DOI":"10.1109\/WACV48630.2021.00301"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Dai, X., Singh, B., Ng, J.Y.H., and Davis, L. (2019, January 7\u201311). Tan: Temporal aggregation network for dense multi-label action recognition. Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), Honolulu, HI, USA.","DOI":"10.1109\/WACV.2019.00022"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zhang, C., Wu, J., and Li, Y. (2022). Actionformer: Localizing moments of actions with transformers. arXiv.","DOI":"10.1007\/978-3-031-19772-7_29"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Tan, J., Tang, J., Wang, L., and Wu, G. (2021, January 10\u201317). Relaxed transformer decoders for direct action proposal generation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.01327"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Dai, R., Das, S., Kahatapitiya, K., Ryoo, M.S., and Bremond, F. (2022, January 21\u201324). MS-TCT: Multi-Scale Temporal ConvTransformer for Action Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01941"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"He, L., and Todorovic, S. (2022, January 21\u201324). DESTR: Object Detection with Split Transformer. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00916"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Li, Y., Wu, C.Y., Fan, H., Mangalam, K., Xiong, B., Malik, J., and Feichtenhofer, C. (2022, January 21\u201324). MViTv2: Improved Multiscale Vision Transformers for Classification and Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00476"},{"key":"ref_9","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_10","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Chao, Y.W., Vijayanarasimhan, S., Seybold, B., Ross, D.A., Deng, J., and Sukthankar, R. (2018, January 18\u201323). Rethinking the faster r-cnn architecture for temporal action localization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00124"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"8535","DOI":"10.1109\/TIP.2020.3016486","article-title":"Revisiting anchor mechanisms for temporal action localization","volume":"29","author":"Yang","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_13","unstructured":"Chen, G., Zheng, Y.D., Wang, L., and Lu, T. (March, January 22). DCAN: Improving temporal action detection via dual context aggregation. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"652","DOI":"10.1109\/TPAMI.2019.2938758","article-title":"Res2net: A new multi-scale backbone architecture","volume":"43","author":"Gao","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2017, January 21\u201326). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Xia, K., Wang, L., Zhou, S., Zheng, N., and Tang, W. (2022, January 19\u201320). Learning To Refactor Action and Co-Occurrence Features for Temporal Action Localization. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01351"},{"key":"ref_17","unstructured":"Simonyan, K., and Zisserman, A. (2014). Two-stream convolutional networks for action recognition in videos. Adv. Neural Inf. Process. Syst., 27."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Caba Heilbron, F., Escorcia, V., Ghanem, B., and Carlos Niebles, J. (2015, January 7). Activitynet: A large-scale video benchmark for human activity understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298698"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cviu.2016.10.018","article-title":"The THUMOS challenge on action recognition for videos \u201cin the wild\u201d","volume":"155","author":"Idrees","year":"2017","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_20","unstructured":"Shou, Z., Wang, D., and Chang, S.F. (July, January 26). Temporal action localization in untrimmed videos via multi-stage cnns. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Gao, J., Yang, Z., Chen, K., Sun, C., and Nevatia, R. (2017, January 22\u201329). Turn tap: Temporal unit regression network for temporal action proposals. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.392"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Gao, J., Chen, K., and Nevatia, R. (2018, January 8\u201314). Ctap: Complementary temporal action proposal generation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01216-8_5"},{"key":"ref_23","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Adv. Neural Inf. Process. Syst., 28."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Xu, H., Das, A., and Saenko, K. (2017, January 22\u201329). R-c3d: Region convolutional 3d network for temporal activity detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.617"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Long, F., Yao, T., Qiu, Z., Tian, X., Luo, J., and Mei, T. (2019, January 15\u201320). Gaussian temporal awareness networks for action localization. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00043"},{"key":"ref_26","unstructured":"Liu, Q., and Wang, Z. (2020, January 7\u201312). Progressive boundary refinement network for temporal action detection. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lin, C., Xu, C., Luo, D., Wang, Y., Tai, Y., Wang, C., Li, J., Huang, F., and Fu, Y. (2021, January 19\u201325). Learning salient boundary feature for anchor-free temporal action localization. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00333"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5427","DOI":"10.1109\/TIP.2022.3195321","article-title":"End-to-end temporal action detection with transformer","volume":"31","author":"Liu","year":"2022","journal-title":"IEEE Trans. Image Process."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zhao, Y., Xiong, Y., Wang, L., Wu, Z., Tang, X., and Lin, D. (2017, January 22\u201329). Temporal action detection with structured segment networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.317"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Lin, T., Zhao, X., Su, H., Wang, C., and Yang, M. (2018, January 8\u201314). Bsn: Boundary sensitive network for temporal action proposal generation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01225-0_1"},{"key":"ref_31","unstructured":"Lin, T., Liu, X., Li, X., Ding, E., and Wen, S. (November, January 27). Bmn: Boundary-matching network for temporal action proposal generation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Su, H., Gan, W., Wu, W., Qiao, Y., and Yan, J. (2021, January 2\u20139). Bsn++: Complementary boundary regressor with scale-balanced relation modeling for temporal action proposal generation. Proceedings of the AAAI Conference on Artificial Intelligence, Online.","DOI":"10.1609\/aaai.v35i3.16363"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhao, P., Xie, L., Ju, C., Zhang, Y., Wang, Y., and Tian, Q. (2020, January 23\u201328). Bottom-up temporal action localization with mutual regularization. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58598-3_32"},{"key":"ref_34","unstructured":"Zeng, R., Huang, W., Tan, M., Rong, Y., Zhao, P., Huang, J., and Gan, C. (November, January 27). Graph convolutional networks for temporal action localization. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Xu, M., Zhao, C., Rojas, D.S., Thabet, A., and Ghanem, B. (2020, January 13\u201319). G-tad: Sub-graph localization for temporal action detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01017"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Bodla, N., Singh, B., Chellappa, R., and Davis, L.S. (2017, January 22\u201329). Soft-NMS\u2013improving object detection with one line of code. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.593"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Carreira, J., and Zisserman, A. (2017, January 21\u201326). Quo vadis, action recognition? A new model and the kinetics dataset. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.502"},{"key":"ref_39","unstructured":"Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., and Natsev, P. (2017). The kinetics human action video dataset. arXiv."},{"key":"ref_40","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., and Paluri, M. (2018, January 18\u201323). A closer look at spatiotemporal convolutions for action recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00675"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Alwassel, H., Giancola, S., and Ghanem, B. (2021, January 10\u201317). Tsp: Temporally-sensitive pretraining of video encoders for localization tasks. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/ICCVW54120.2021.00356"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Liu, X., Bai, S., and Bai, X. (2022, January 19\u201320). An Empirical Study of End-to-End Temporal Action Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01938"},{"key":"ref_44","unstructured":"Yang, H., Wu, W., Wang, L., Jin, S., Xia, B., Yao, H., and Huang, H. (March, January 22). Temporal Action Proposal Generation with Background Constraint. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Zhao, C., Thabet, A.K., and Ghanem, B. (2021, January 10\u201317). Video self-stitching graph network for temporal action localization. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.01340"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Sridhar, D., Quader, N., Muralidharan, S., Li, Y., Dai, P., and Lu, J. (2021, January 10\u201317). Class semantics-based attention for action detection. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.01348"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Liu, X., Hu, Y., Bai, S., Ding, F., Bai, X., and Torr, P.H. (2021, January 19\u201325). Multi-shot temporal event localization: A benchmark. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01241"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Bai, Y., Wang, Y., Tong, Y., Yang, Y., Liu, Q., and Liu, J. (2020, January 23\u201328). Boundary content graph neural network for temporal action proposal generation. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58604-1_8"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Qing, Z., Su, H., Gan, W., Wang, D., Wu, W., Wang, X., Qiao, Y., Yan, J., Gao, C., and Sang, N. (2021, January 19\u201325). Temporal context aggregation network for temporal action proposal refinement. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00055"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/21\/8396\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:08:58Z","timestamp":1760144938000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/21\/8396"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,1]]},"references-count":49,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["s22218396"],"URL":"https:\/\/doi.org\/10.3390\/s22218396","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,1]]}}}