{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,10,31]],"date-time":"2024-10-31T04:18:24Z","timestamp":1730348304177,"version":"3.28.0"},"reference-count":30,"publisher":"National Library of Serbia","issue":"4","license":[{"start":{"date-parts":[[2024,1,1]],"date-time":"2024-01-01T00:00:00Z","timestamp":1704067200000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["ComSIS","COMPUT SCI INF SYST","COMPUT SCI INFORM SY","COMPUTER SCI INFORM","COMSIS J"],"published-print":{"date-parts":[[2024]]},"abstract":"<jats:p>This paper introduces spatio-temporal-based multi-level aggregation network (ST-MANet) for action recognition. It utilizes the correlations between different spatial positions and the correlations between different temporal positions on the feature map to explore long-range spatial and temporal dependencies, respectively, generating the spatial and temporal attention map that assigns different weights to features at different spatial and temporal locations. Additionally, a multi-scale approach is introduced, proposing a multi-scale behavior recognition framework that models various visual rhythms while capturing multi-scale spatiotemporal information. A spatial diversity constraint is then proposed, encouraging spatial attention maps at different scales to focus on distinct areas. This ensures a greater emphasis on spatial information unique to each scale, thereby incorporating more diverse spatial information into multi-scale features. Finally, ST-MANet is compared with existing approaches, demonstrating high accuracy on the three datasets.<\/jats:p>","DOI":"10.2298\/csis240418060w","type":"journal-article","created":{"date-parts":[[2024,9,24]],"date-time":"2024-09-24T13:09:08Z","timestamp":1727183348000},"page":"1823-1843","source":"Crossref","is-referenced-by-count":0,"title":["Spatio-temporal-based multi-level aggregation network for physical action recognition"],"prefix":"10.2298","volume":"21","author":[{"given":"Yuhang","family":"Wang","sequence":"first","affiliation":[{"name":"School of Physical Education, Harbin University, Harbin, Heilongjiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1078","reference":[{"key":"ref1","doi-asserted-by":"crossref","unstructured":"Carreira, J., Zisserman, A.: Quo vadis, action recognition? a new model and the kinetics dataset. In: proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 6299-6308 (2017)","DOI":"10.1109\/CVPR.2017.502"},{"key":"ref2","doi-asserted-by":"crossref","unstructured":"Chai, W., Jiang, Z., Hwang, J.N., Wang, G.: Global adaptation meets local generalization: Unsupervised domain adaptation for 3d human pose estimation. In: ICCV. pp. 14655-14665 (2023)","DOI":"10.1109\/ICCV51070.2023.01347"},{"key":"ref3","doi-asserted-by":"crossref","unstructured":"Chen, J., Ho, C.M.: Mm-vit: Multi-modal video transformer for compressed video action recognition. In: Proceedings of the IEEE\/CVF winter conference on applications of computer vision. pp. 1910-1921 (2022)","DOI":"10.1109\/WACV51458.2022.00086"},{"key":"ref4","doi-asserted-by":"crossref","unstructured":"Feichtenhofer, C.: X3d: Expanding architectures for efficient video recognition. In: computer vision and pattern recognition. pp. 203-213 (2020)","DOI":"10.1109\/CVPR42600.2020.00028"},{"key":"ref5","doi-asserted-by":"crossref","unstructured":"Feichtenhofer, C., Fan, H., Malik, J., He, K.: Slowfast networks for video recognition. In: international conference on computer vision. pp. 6202-6211 (2019)","DOI":"10.1109\/ICCV.2019.00630"},{"key":"ref6","doi-asserted-by":"crossref","unstructured":"Gao, J., Liu, M., Li, P., Zhang, J., Chen, Z.: Deep multiview adaptive clustering with semantic invariance. IEEE Transactions on Neural Networks and Learning Systems (2023)","DOI":"10.1109\/TNNLS.2023.3265699"},{"key":"ref7","doi-asserted-by":"crossref","unstructured":"Guan, X., Yang, Y., Li, J., Xu, X., Shen, H.T.: Mind the remainder: taylor\u2019s theorem view on recurrent neural networks. IEEE Transactions on Neural Networks and Learning Systems 33(4), 1507-1519 (2021)","DOI":"10.1109\/TNNLS.2020.3042537"},{"key":"ref8","doi-asserted-by":"crossref","unstructured":"Guo, C., Fan, B., Zhang, Q., Xiang, S., Pan, C.: Augfpn: Improving multi-scale feature learning for object detection. In: computer vision and pattern recognition. pp. 12595-12604 (2020)","DOI":"10.1109\/CVPR42600.2020.01261"},{"key":"ref9","doi-asserted-by":"crossref","unstructured":"Gupta, P., Thatipelli, A., Aggarwal, A., Maheshwari, S., Trivedi, N., Das, S., Sarvadevabhatla, R.K.: Quo vadis, skeleton action recognition? International Journal of Computer Vision 129(7), 2097-2112 (2021)","DOI":"10.1007\/s11263-021-01470-y"},{"key":"ref10","doi-asserted-by":"crossref","unstructured":"He, Z., Lv, J., Fang, S.: Representation modeling learning with multi-domain decoupling for unsupervised skeleton-based action recognition. Neurocomputing p. 127495 (2024)","DOI":"10.1016\/j.neucom.2024.127495"},{"key":"ref11","doi-asserted-by":"crossref","unstructured":"Huo, J., Cai, H., Meng, Q.: Independent dual graph attention convolutional network for skeleton-based action recognition. Neurocomputing p. 127496 (2024)","DOI":"10.1016\/j.neucom.2024.127496"},{"key":"ref12","doi-asserted-by":"crossref","unstructured":"Jiang, B., Wang, M., Gan, W., Wu, W., Yan, J.: Stm: Spatiotemporal and motion encoding for action recognition. In: the IEEE\/CVF international conference on computer vision. pp. 2000- 2009 (2019)","DOI":"10.1109\/ICCV.2019.00209"},{"key":"ref13","doi-asserted-by":"crossref","unstructured":"Karim, M., Khalid, S., Aleryani, A., Khan, J., Ullah, I., Ali, Z.: Human action recognition systems: A review of the trends and state-of-the-art. IEEE Access (2024)","DOI":"10.1109\/ACCESS.2024.3373199"},{"key":"ref14","unstructured":"Karpathy, A., Joulin, A., Fei-Fei, L.F.: Deep fragment embeddings for bidirectional image sentence mapping. Advances in neural information processing systems 27 (2014)"},{"key":"ref15","doi-asserted-by":"crossref","unstructured":"Li, P., Gao, J., Zhang, J., Jin, S., Chen, Z.: Deep reinforcement clustering. IEEE Transactions on Multimedia (2022)","DOI":"10.1109\/TMM.2022.3233249"},{"key":"ref16","doi-asserted-by":"crossref","unstructured":"Li, Y., Ji, B., Shi, X., Zhang, J., Kang, B., Wang, L.: Tea: Temporal excitation and aggregation for action recognition. In: computer vision and pattern recognition. pp. 909-918 (2020)","DOI":"10.1109\/CVPR42600.2020.00099"},{"key":"ref17","doi-asserted-by":"crossref","unstructured":"Lin, J., Gan, C., Han, S.: Tsm: Temporal shift module for efficient video understanding. In: international conference on computer vision. pp. 7083-7093 (2019)","DOI":"10.1109\/ICCV.2019.00718"},{"key":"ref18","doi-asserted-by":"crossref","unstructured":"Liu, Z., Luo, D.,Wang, Y.,Wang, L., Tai, Y.,Wang, C., Li, J., Huang, F., Lu, T.: Teinet: Towards an efficient architecture for video recognition. In: Proceedings of the AAAI conference on artificial intelligence. vol. 34, pp. 11669-11676 (2020)","DOI":"10.1609\/aaai.v34i07.6836"},{"key":"ref19","unstructured":"Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. Advances in neural information processing systems 27 (2014)"},{"key":"ref20","doi-asserted-by":"crossref","unstructured":"Sun, L., Jia, K., Chen, K., Yeung, D.Y., Shi, B.E., Savarese, S.: Lattice long short-term memory for human action recognition. In: IEEE international conference on computer vision. pp. 2147- 2156 (2017)","DOI":"10.1109\/ICCV.2017.236"},{"key":"ref21","doi-asserted-by":"crossref","unstructured":"Tao, L., Wang, X., Yamasaki, T.: Rethinking motion representation: Residual frames with 3d convnets. IEEE Transactions on Image Processing 30, 9231-9244 (2021)","DOI":"10.1109\/TIP.2021.3124156"},{"key":"ref22","doi-asserted-by":"crossref","unstructured":"Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3d convolutional networks. In: IEEE international conference on computer vision. pp. 4489-4497 (2015)","DOI":"10.1109\/ICCV.2015.510"},{"key":"ref23","doi-asserted-by":"crossref","unstructured":"Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3d convolutional networks. In: IEEE international conference on computer vision. pp. 4489-4497 (2015)","DOI":"10.1109\/ICCV.2015.510"},{"key":"ref24","doi-asserted-by":"crossref","unstructured":"Varol, G., Laptev, I., Schmid, C.: Long-term temporal convolutions for action recognition. IEEE transactions on pattern analysis and machine intelligence 40(6), 1510-1517 (2017)","DOI":"10.1109\/TPAMI.2017.2712608"},{"key":"ref25","doi-asserted-by":"crossref","unstructured":"Wang, L., Tong, Z., Ji, B., Wu, G.: Tdn: Temporal difference networks for efficient action recognition. In: computer vision and pattern recognition. pp. 1895-1904 (2021)","DOI":"10.1109\/CVPR46437.2021.00193"},{"key":"ref26","doi-asserted-by":"crossref","unstructured":"Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., Van Gool, L.: Temporal segment networks: Towards good practices for deep action recognition. In: European conference on computer vision. pp. 20-36 (2016)","DOI":"10.1007\/978-3-319-46484-8_2"},{"key":"ref27","doi-asserted-by":"crossref","unstructured":"Wang, Y., Long, M., Wang, J., Yu, P.S.: Spatiotemporal pyramid network for video action recognition. In: Computer Vision and Pattern Recognition. pp. 1529-1538 (2017)","DOI":"10.1109\/CVPR.2017.226"},{"key":"ref28","doi-asserted-by":"crossref","unstructured":"Yang, C., Xu, Y., Shi, J., Dai, B., Zhou, B.: Temporal pyramid network for action recognition. In: computer vision and pattern recognition. pp. 591-600 (2020)","DOI":"10.1109\/CVPR42600.2020.00067"},{"key":"ref29","doi-asserted-by":"crossref","unstructured":"Yang, H., Yuan, C., Li, B., Du, Y., Xing, J., Hu, W., Maybank, S.J.: Asymmetric 3d convolutional neural networks for action recognition. Pattern recognition 85, 1-12 (2019)","DOI":"10.1016\/j.patcog.2018.07.028"},{"key":"ref30","doi-asserted-by":"crossref","unstructured":"Zhang, D., Dai, X., Wang, Y.F.: Dynamic temporal pyramid network: A closer look at multiscale modeling for activity detection. In: Computer Vision. pp. 712-728 (2019)","DOI":"10.1007\/978-3-030-20870-7_44"}],"container-title":["Computer Science and Information Systems"],"original-title":[],"language":"en","deposited":{"date-parts":[[2024,10,30]],"date-time":"2024-10-30T08:27:16Z","timestamp":1730276836000},"score":1,"resource":{"primary":{"URL":"https:\/\/doiserbia.nb.rs\/Article.aspx?ID=1820-02142400060W"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":30,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024]]}},"URL":"https:\/\/doi.org\/10.2298\/csis240418060w","relation":{},"ISSN":["1820-0214","2406-1018"],"issn-type":[{"type":"print","value":"1820-0214"},{"type":"electronic","value":"2406-1018"}],"subject":[],"published":{"date-parts":[[2024]]}}}