{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T14:08:09Z","timestamp":1778854089912,"version":"3.51.4"},"reference-count":65,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T00:00:00Z","timestamp":1778803200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62495084 and 62473307"],"award-info":[{"award-number":["62495084 and 62473307"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2026,5,31]]},"abstract":"<jats:p>Group activity recognition involves detecting the collective actions performed by a group of individuals, where identifying the key actors and key frames is crucial for understanding the group\u2019s behavior. To tackle this challenge, we propose a spatio-temporal reasoning framework that leverages key instances. Our key instance identification module effectively detects key roles and frames from video sequences, while a graph-based reasoning model dynamically aggregates the features of related actors. We extract joint features and RGB features from video sequences, and these are fused using our multi-modal fusion TCT module, which improves the representation power of the original features. To better understand group activity through spatio-temporal correlations, we further utilize an enhanced cross-transformer module for spatio-temporal synchronous reasoning, considering both time and space dimensions. Our method has been tested on two public datasets, showing that it achieves high accuracy and surpasses many state-of-the-art approaches.<\/jats:p>","DOI":"10.1145\/3805798","type":"journal-article","created":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T13:46:18Z","timestamp":1775137578000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Key Instance-Based Spatio-Temporal Network for Group Activity Recognition"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5741-5280","authenticated-orcid":false,"given":"Yaochen","family":"Li","sequence":"first","affiliation":[{"name":"School of Software Engineering, Xi\u2019an Jiaotong University, Xi\u2019an, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-1593-3676","authenticated-orcid":false,"given":"Haoting","family":"He","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Xi\u2019an Jiaotong University, Xi\u2019an, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-2628-3457","authenticated-orcid":false,"given":"Yutong","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Xi\u2019an Jiaotong University, Xi\u2019an, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-8121-5987","authenticated-orcid":false,"given":"Gaojie","family":"Li","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Xi\u2019an Jiaotong University, Xi\u2019an, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-3114-6800","authenticated-orcid":false,"given":"Junyi","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Xi\u2019an Jiaotong University, Xi\u2019an, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-6931-3638","authenticated-orcid":false,"given":"Yuehu","family":"Liu","sequence":"additional","affiliation":[{"name":"Institute of Artificial Intelligence and Robotics, Xi\u2019an Jiaotong University, Xi\u2019an, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,5,15]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"572","volume-title":"Proceedings of the 13th European Conference on Computer Vision (ECCV \u201914), Part VI","author":"Amer Mohamed Rabie","year":"2014","unstructured":"Mohamed Rabie Amer, Peng Lei, and Sinisa Todorovic. 2014. Hirf: Hierarchical random field for collective activity recognition in videos. In Proceedings of the 13th European Conference on Computer Vision (ECCV \u201914), Part VI. Springer, 572\u2013585."},{"key":"e_1_3_1_3_2","doi-asserted-by":"crossref","first-page":"7884","DOI":"10.1109\/CVPR.2019.00808","volume-title":"2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Azar S.","year":"2019","unstructured":"S. Azar, M. Atigh, A. Nickabadi, and A. Alahi. 2019. Convolutional relational machine for group activity recognition. In 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, 7884\u20137893. DOI: 10.1109\/CVPR.2019.00808"},{"key":"e_1_3_1_4_2","doi-asserted-by":"crossref","first-page":"3425","DOI":"10.1109\/CVPR.2017.365","volume-title":"2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Bagautdinov Timur","year":"2017","unstructured":"Timur Bagautdinov, Alexandre Alahi, Fran\u00e7ois Fleuret, Pascal Fua, and Silvio Savarese. 2017. Social scene understanding: End-to-end multi-person action localization and collective activity recognition. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3425\u20133434. DOI: 10.1109\/CVPR.2017.365"},{"key":"e_1_3_1_5_2","first-page":"3286","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Bello Irwan","year":"2019","unstructured":"Irwan Bello, Barret Zoph, Ashish Vaswani, Jonathon Shlens, and Quoc V. Le. 2019. Attention augmented convolutional networks. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, 3286\u20133295."},{"key":"e_1_3_1_6_2","doi-asserted-by":"crossref","first-page":"4724","DOI":"10.1109\/CVPR.2017.502","volume-title":"2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Carreira Jo\u00e3o","year":"2017","unstructured":"Jo\u00e3o Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? A new model and the kinetics dataset. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4724\u20134733. DOI: 10.1109\/CVPR.2017.502"},{"key":"e_1_3_1_7_2","doi-asserted-by":"crossref","first-page":"5158","DOI":"10.1109\/CVPRW59228.2023.00544","volume-title":"2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)","author":"Chappa N.","year":"2023","unstructured":"N. Chappa, P. Nguyen, A. H. Nelson, H. Seo, X. Li, P. Dobbs, and K. Luu. 2023. SPARTAN: Self-supervised spatiotemporal transformers approach to group activity recognition. In 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE Computer Society, Los Alamitos, CA, 5158\u20135168. DOI: 10.1109\/CVPRW59228.2023.00544"},{"key":"e_1_3_1_8_2","first-page":"215","volume-title":"Proceedings of the 12th European Conference on Computer Vision (ECCV \u201912), Part IV","author":"Choi Wongun","year":"2012","unstructured":"Wongun Choi and Silvio Savarese. 2012. A unified framework for multi-target tracking and collective activity recognition. In Proceedings of the 12th European Conference on Computer Vision (ECCV \u201912), Part IV. Springer, 215\u2013230."},{"key":"e_1_3_1_9_2","doi-asserted-by":"crossref","first-page":"1282","DOI":"10.1109\/ICCVW.2009.5457461","volume-title":"2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops \u201909)","author":"Choi Wongun","year":"2009","unstructured":"Wongun Choi, Khuram Shahid, and Silvio Savarese. 2009. What are they doing? Collective activity classification using spatio-temporal relationship among people. In 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops \u201909), 1282\u20131289. DOI: 10.1109\/ICCVW.2009.5457461"},{"key":"e_1_3_1_10_2","first-page":"3273","volume-title":"Conference on Computer Vision and Pattern Recognition 2011","author":"Choi Wongun","year":"2011","unstructured":"Wongun Choi, Khuram Shahid, and Silvio Savarese. 2011. Learning context for collective activity recognition. In Conference on Computer Vision and Pattern Recognition 2011. IEEE, 3273\u20133280."},{"key":"e_1_3_1_11_2","first-page":"4772","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Deng Zhiwei","year":"2016","unstructured":"Zhiwei Deng, Arash Vahdat, Hexiang Hu, and Greg Mori. 2016. Structure inference machines: Recurrent neural networks for analyzing relations in group activity recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4772\u20134781."},{"key":"e_1_3_1_12_2","first-page":"177","volume-title":"Computer Vision (ECCV \u201920)","author":"Ehsanpour Mahsa","year":"2020","unstructured":"Mahsa Ehsanpour, Alireza Abedin, Fatemeh Saleh, Javen Shi, Ian Reid, and Hamid Rezatofighi. 2020. Joint learning of social groups, individuals action and Sub-group activities in videos. In Computer Vision (ECCV \u201920). Springer International Publishing, Cham, 177\u2013195."},{"key":"e_1_3_1_13_2","doi-asserted-by":"crossref","first-page":"836","DOI":"10.1109\/CVPR42600.2020.00092","volume-title":"2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Gavrilyuk Kirill","year":"2020","unstructured":"Kirill Gavrilyuk, Ryan Sanford, Mehrsan Javan, and Cees G. M. Snoek. 2020. Actor-transformers for group activity recognition. In 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 836\u2013845. DOI: 10.1109\/CVPR42600.2020.00092"},{"key":"e_1_3_1_14_2","first-page":"2980","volume-title":"2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Han Mingfei","year":"2022","unstructured":"Mingfei Han, David Junhao Zhang, Yali Wang, Rui Yan, Lina Yao, Xiaojun Chang, and Yu Qiao. 2022. Dual-AI: Dual-path actor interaction learning for group activity recognition. In 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2980\u20132989. DOI: 10.1109\/CVPR52688.2022.00300"},{"key":"e_1_3_1_15_2","first-page":"977","volume-title":"2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Hu Guyue","year":"2020","unstructured":"Guyue Hu, Bo Cui, Yuan He, and Shan Yu. 2020. Progressive relation learning for group activity recognition. In 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 977\u2013986. DOI: 10.1109\/CVPR42600.2020.00106"},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","unstructured":"Yilei Hua Wen Lan Wu Ce Zheng Aidong Lu Mengyuan Liu Chen Chen and Shiqian Wu. 2023. Part aware contrastive learning for self-supervised action recognition. arXiv:2305.00666. Retrieved from https:\/\/arxiv.org\/abs\/2305.00666","DOI":"10.24963\/ijcai.2023\/95"},{"key":"e_1_3_1_17_2","doi-asserted-by":"crossref","first-page":"111104","DOI":"10.1016\/j.patcog.2024.111104","article-title":"Spatio-temporal interactive reasoning model for multi-group activity recognition","volume":"159","author":"Huang Jianglan","year":"2025","unstructured":"Jianglan Huang, Lindong Li, Linbo Qing, Wang Tang, Pingyu Wang, Li Guo, and Yonghong Peng. 2025. Spatio-temporal interactive reasoning model for multi-group activity recognition. Pattern Recognition 159 (2025), 111104.","journal-title":"Pattern Recognition"},{"key":"e_1_3_1_18_2","first-page":"2257","volume-title":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence","author":"Huang Jie","year":"2018","unstructured":"Jie Huang, Wengang Zhou, Qilin Zhang, Houqiang Li, and Weiping Li. 2018. Video-based sign language recognition without temporal segmentation. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2257\u20132264."},{"key":"e_1_3_1_19_2","first-page":"721","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV)","author":"Ibrahim Mostafa S.","year":"2018","unstructured":"Mostafa S. Ibrahim and Greg Mori. 2018. Hierarchical relational networks for group activity recognition and retrieval. In Proceedings of the European Conference on Computer Vision (ECCV), 721\u2013736."},{"key":"e_1_3_1_20_2","doi-asserted-by":"crossref","first-page":"1971","DOI":"10.1109\/CVPR.2016.217","volume-title":"2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Ibrahim Mostafa S.","year":"2016","unstructured":"Mostafa S. Ibrahim, Srikanth Muralidharan, Zhiwei Deng, Arash Vahdat, and Greg Mori. 2016. A hierarchical deep temporal model for group activity recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1971\u20131980. DOI: 10.1109\/CVPR.2016.217"},{"key":"e_1_3_1_21_2","doi-asserted-by":"crossref","first-page":"2555","DOI":"10.1109\/CVPR.2013.330","volume-title":"2013 IEEE Conference on Computer Vision and Pattern Recognition","author":"Jain Mihir","year":"2013","unstructured":"Mihir Jain, Herv\u00e9 J\u00e9gou, and Patrick Bouthemy. 2013. Better exploiting motion for better action recognition. In 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2555\u20132562. DOI: 10.1109\/CVPR.2013.330"},{"key":"e_1_3_1_22_2","first-page":"2000","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Jiang Boyuan","year":"2019","unstructured":"Boyuan Jiang, MengMeng Wang, Weihao Gan, Wei Wu, and Junjie Yan. 2019. Stm: Spatiotemporal and motion encoding for action recognition. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, 2000\u20132009."},{"key":"e_1_3_1_23_2","doi-asserted-by":"crossref","first-page":"108412","DOI":"10.1016\/j.engappai.2024.108412","article-title":"Unveiling group activity recognition: Leveraging local\u2013global context-aware graph reasoning for enhanced actor\u2013scene interactions","volume":"133","author":"Jiang Xue","year":"2024","unstructured":"Xue Jiang, Linbo Qing, Jianglan Huang, Li Guo, and Yonghong Peng. 2024. Unveiling group activity recognition: Leveraging local\u2013global context-aware graph reasoning for enhanced actor\u2013scene interactions. Engineering Applications of Artificial Intelligence 133 (2024), 108412.","journal-title":"Engineering Applications of Artificial Intelligence"},{"key":"e_1_3_1_24_2","doi-asserted-by":"crossref","first-page":"20051","DOI":"10.1109\/CVPR52688.2022.01945","volume-title":"2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Kim D.","year":"2022","unstructured":"D. Kim, J. Lee, M. Cho, and S. Kwak. 2022. Detector-Free weakly supervised group activity recognition. In 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, 20051\u201320061. DOI: 10.1109\/CVPR52688.2022.01945"},{"key":"e_1_3_1_25_2","first-page":"1","volume-title":"2023 IEEE International Conference on Visual Communications and Image Processing (VCIP)","author":"Kim Donguk","year":"2023","unstructured":"Donguk Kim, Sumin Lee, Sangmin Woo, Jinyoung Park, Muhammad Adi Nugroho, and Changick Kim. 2023. Multi-modal social group activity recognition in panoramic scene. In 2023 IEEE International Conference on Visual Communications and Image Processing (VCIP), 1\u20135. DOI: 10.1109\/VCIP59821.2023.10402675"},{"key":"e_1_3_1_26_2","unstructured":"Thomas N. Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv:1609.02907. Retrieved from https:\/\/arxiv.org\/abs\/1609.02907"},{"key":"e_1_3_1_27_2","doi-asserted-by":"crossref","first-page":"1328","DOI":"10.1109\/ICASSP.2018.8461770","volume-title":"2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","author":"Kong Longteng","year":"2018","unstructured":"Longteng Kong, Jie Qin, Di Huang, Yunhong Wang, and Luc Van Gool. 2018. Hierarchical attention and context modeling for group activity recognition. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1328\u20131332."},{"key":"e_1_3_1_28_2","doi-asserted-by":"crossref","first-page":"1354","DOI":"10.1109\/CVPR.2012.6247821","volume-title":"2012 IEEE Conference on Computer Vision and Pattern Recognition","author":"Lan Tian","year":"2012","unstructured":"Tian Lan, Leonid Sigal, and Greg Mori. 2012. Social roles in hierarchical models for human activity recognition. In 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1354\u20131361."},{"issue":"8","key":"e_1_3_1_29_2","doi-asserted-by":"crossref","first-page":"1549","DOI":"10.1109\/TPAMI.2011.228","article-title":"Discriminative latent models for recognizing contextual group activities","volume":"34","author":"Lan Tian","year":"2011","unstructured":"Tian Lan, Yang Wang, Weilong Yang, Stephen N. Robinovitch, and Greg Mori. 2011. Discriminative latent models for recognizing contextual group activities. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 8 (2011), 1549\u20131562.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_1_30_2","first-page":"13668","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Li Shuaicheng","year":"2021","unstructured":"Shuaicheng Li, Qianggang Cao, Lingbo Liu, Kunlin Yang, Shinan Liu, Jun Hou, and Shuai Yi. 2021. GroupFormer: Group activity recognition with clustered spatial-temporal transformer. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, 13668\u201313677."},{"key":"e_1_3_1_31_2","first-page":"22","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Li X.","year":"2017","unstructured":"X. Li, Choo Chuah, and M. Sbgar. 2017. Semantics based group activity recognition. In Proceedings of the IEEE International Conference on Computer Vision, 22\u201329."},{"key":"e_1_3_1_32_2","first-page":"909","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Li Yan","year":"2020","unstructured":"Yan Li, Bin Ji, Xintian Shi, Jianguo Zhang, Bin Kang, and Limin Wang. 2020. Tea: Temporal excitation and aggregation for action recognition. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 909\u2013918."},{"key":"e_1_3_1_33_2","first-page":"7082","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Lin Ji","year":"2019","unstructured":"Ji Lin, Chuang Gan, and Song Han. 2019. TSM: Temporal shift module for efficient video understanding. In Proceedings of the IEEE International Conference on Computer Vision, 7082\u20137092."},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2021.108360"},{"key":"e_1_3_1_35_2","first-page":"71","volume-title":"Computer Vision (ECCV \u201920)","author":"Adhi Pramono Rizard Renanda","year":"2020","unstructured":"Rizard Renanda Adhi Pramono, Yie Tarng Chen, and Wen Hsien Fang. 2020. Empowering relational network by self-attention augmented conditional random fields for group activity recognition. In Computer Vision (ECCV \u201920). Springer International Publishing, Cham, 71\u201390."},{"key":"e_1_3_1_36_2","doi-asserted-by":"crossref","first-page":"8184","DOI":"10.1109\/TIP.2021.3113570","article-title":"Relational reasoning for group activity recognition via self-attention augmented conditional random field","volume":"30","author":"Adhi Pramono Rizard Renanda","year":"2021","unstructured":"Rizard Renanda Adhi Pramono, Wen-Hsien Fang, and Yie-Tarng Chen. 2021. Relational reasoning for group activity recognition via self-attention augmented conditional random field. IEEE Transactions on Image Processing 30 (2021), 8184\u20138199.","journal-title":"IEEE Transactions on Image Processing"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2019.2894161"},{"key":"e_1_3_1_38_2","doi-asserted-by":"crossref","first-page":"3043","DOI":"10.1109\/CVPR.2016.332","volume-title":"2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Ramanathan Vignesh","year":"2016","unstructured":"Vignesh Ramanathan, Jonathan Huang, Sami Abu-El-Haija, Alexander Gorban, Kevin Murphy, and Li Fei-Fei. 2016. Detecting events and key actors in multi-person videos. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3043\u20133053. DOI: 10.1109\/CVPR.2016.332"},{"key":"e_1_3_1_39_2","volume-title":"Advances in Neural Information Processing Systems (NeurIPS)","author":"Ryoo Michael S.","year":"2021","unstructured":"Michael S. Ryoo, A. J. Piergiovanni, Anurag Arnab, Mostafa Dehghani, and Anelia Angelova. 2021. TokenLearner: Adaptive space-time tokenization for videos. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 34."},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3068335"},{"key":"e_1_3_1_41_2","first-page":"5523","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Shu Tianmin","year":"2017","unstructured":"Tianmin Shu, Sinisa Todorovic, and Song-Chun Zhu. 2017. Cern: Confidence-energy recurrent network for group activity recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5523\u20135531."},{"issue":"3","key":"e_1_3_1_42_2","first-page":"1110","article-title":"Hierarchical long short-term concurrent memory for human interaction recognition","volume":"43","author":"Shu Xiangbo","year":"2019","unstructured":"Xiangbo Shu, Jinhui Tang, Guo-Jun Qi, Wei Liu, and Jian Yang. 2019. Hierarchical long short-term concurrent memory for human interaction recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 3 (2019), 1110\u20131118.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_1_43_2","first-page":"568","volume-title":"Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS\u201914)","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Two-stream convolutional networks for action recognition in videos. In Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS\u201914). MIT Press, Cambridge, MA, 568\u2013576."},{"key":"e_1_3_1_44_2","first-page":"4597","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Sun Lin","year":"2015","unstructured":"Lin Sun, Kui Jia, Dit-Yan Yeung, and Bertram E. Shi. 2015. Human action recognition using factorized spatio-temporal convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision, 4597\u20134605."},{"key":"e_1_3_1_45_2","first-page":"1","volume-title":"2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Szegedy C.","year":"2015","unstructured":"C. Szegedy, Wei Liu, Yangqing Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. 2015. Going deeper with convolutions. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, 1\u20139. DOI: 10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_1_46_2","first-page":"19","volume-title":"Proceedings of the 17th European Conference on Computer Vision (ECCV \u201922), Part IV","author":"Tamura Masato","year":"2022","unstructured":"Masato Tamura, Rahul Vishwakarma, and Ravigopal Vennelakanti. 2022. Hunting group clues with transformers for social group activity recognition. In Proceedings of the 17th European Conference on Computer Vision (ECCV \u201922), Part IV. Springer-Verlag, Berlin, 19\u201335. DOI: 10.1007\/978-3-031-19772-7_2"},{"key":"e_1_3_1_47_2","first-page":"19","volume-title":"Computer Vision (ECCV \u201922)","author":"Tamura Masato","year":"2022","unstructured":"Masato Tamura, Rahul Vishwakarma, and Ravigopal Vennelakanti. 2022. Hunting group clues with transformers for social group activity recognition. In Computer Vision (ECCV \u201922). Springer Nature Switzerland, Cham, 19\u201335."},{"issue":"2","key":"e_1_3_1_48_2","doi-asserted-by":"crossref","first-page":"636","DOI":"10.1109\/TPAMI.2019.2928540","article-title":"Coherence constrained graph LSTM for group activity recognition","volume":"44","author":"Tang Jinhui","year":"2019","unstructured":"Jinhui Tang, Xiangbo Shu, Rui Yan, and Liyan Zhang. 2019. Coherence constrained graph LSTM for group activity recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 2 (2019), 636\u2013647.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_1_49_2","doi-asserted-by":"crossref","first-page":"1283","DOI":"10.1145\/3240508.3240576","volume-title":"Proceedings of the 26th ACM International Conference on Multimedia","author":"Tang Yansong","year":"2018","unstructured":"Yansong Tang, Zian Wang, Peiyang Li, Jiwen Lu, Ming Yang, and Jie Zhou. 2018. Mining semantics-preserving attention for group activity recognition. In Proceedings of the 26th ACM International Conference on Multimedia, 1283\u20131291."},{"key":"e_1_3_1_50_2","doi-asserted-by":"crossref","first-page":"4489","DOI":"10.1109\/ICCV.2015.510","volume-title":"2015 IEEE International Conference on Computer Vision (ICCV)","author":"Tran Du","year":"2015","unstructured":"Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. 2015. Learning spatiotemporal features with 3D convolutional networks. In 2015 IEEE International Conference on Computer Vision (ICCV), 4489\u20134497. DOI: 10.1109\/ICCV.2015.510"},{"key":"e_1_3_1_51_2","first-page":"5998","article-title":"Attention is all you need","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30 (2017), 5998\u20136008.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_52_2","doi-asserted-by":"crossref","first-page":"111710","DOI":"10.1016\/j.patcog.2025.111710","article-title":"Frequency-aware self-supervised group activity recognition with skeleton sequences","author":"Wang Guoquan","year":"2025","unstructured":"Guoquan Wang, Mengyuan Liu, Hong Liu, Jinyan Zhang, Peini Guo, Ruijia Fan, and Siyu Chen. 2025. Frequency-aware self-supervised group activity recognition with skeleton sequences. Pattern Recognition 167 (2025), 111710.","journal-title":"Pattern Recognition"},{"key":"e_1_3_1_53_2","first-page":"20","volume-title":"European Conference on Computer Vision","author":"Wang Limin","year":"2016","unstructured":"Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool. 2016. Temporal segment networks: Towards good practices for deep action recognition. In European Conference on Computer Vision. Springer, 20\u201336."},{"key":"e_1_3_1_54_2","first-page":"3048","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Wang Minsi","year":"2017","unstructured":"Minsi Wang, Bingbing Ni, and Xiaokang Yang. 2017. Recurrent modeling of interaction context for collective activity recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3048\u20133056."},{"key":"e_1_3_1_55_2","first-page":"7794","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Wang Xiaolong","year":"2018","unstructured":"Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. 2018. Non-local neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7794\u20137803."},{"key":"e_1_3_1_56_2","first-page":"13214","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Wang Zhengwei","year":"2021","unstructured":"Zhengwei Wang, Qi She, and Aljosa Smolic. 2021. Action-net: Multipath excitation for action recognition. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 13214\u201313223."},{"key":"e_1_3_1_57_2","first-page":"9956","volume-title":"2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201919)","author":"Wu Jianchao","year":"2019","unstructured":"Jianchao Wu, Limin Wang, Li Wang, Jie Guo, and Gangshan Wu. 2019. Learning actor relation graphs for group activity recognition. In 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201919), 9956\u20139966. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:128358806"},{"key":"e_1_3_1_58_2","doi-asserted-by":"crossref","first-page":"65689","DOI":"10.1109\/ACCESS.2020.2979742","article-title":"Group activity recognition by using effective multiple modality relation representation with temporal-spatial attention","volume":"8","author":"Xu Dezhong","year":"2020","unstructured":"Dezhong Xu, Heng Fu, Lifang Wu, Meng Jian, Dong Wang, and Xu Liu. 2020. Group activity recognition by using effective multiple modality relation representation with temporal-spatial attention. IEEE Access 8 (2017), 65689\u201365698.","journal-title":"IEEE Access"},{"key":"e_1_3_1_59_2","first-page":"5783","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Xu Huijuan","year":"2017","unstructured":"Huijuan Xu, Abir Das, and Kate Saenko. 2017. R-c3d: Region convolutional 3d network for temporal activity detection. In Proceedings of the IEEE International Conference on Computer Vision, 5783\u20135792."},{"key":"e_1_3_1_60_2","doi-asserted-by":"crossref","first-page":"1292","DOI":"10.1145\/3240508.3240572","volume-title":"Proceedings of the 26th ACM International Conference on Multimedia (MM \u201918)","author":"Yan Rui","year":"2018","unstructured":"Rui Yan, Jinhui Tang, Xiangbo Shu, Zechao Li, and Qi Tian. 2018. Participation-Contributed temporal dynamic model for group activity recognition. In Proceedings of the 26th ACM International Conference on Multimedia (MM \u201918). ACM, New York, NY, 1292\u20131300. DOI: 10.1145\/3240508.3240572"},{"key":"e_1_3_1_61_2","doi-asserted-by":"crossref","unstructured":"Rui Yan Lingxi Xie Jinhui Tang Xiangbo Shu and Qi Tian. 2020. Social adaptive module for weakly-supervised group activity recognition. arXiv:2007.09470. Retrieved from https:\/\/arxiv.org\/abs\/2007.09470","DOI":"10.1007\/978-3-030-58598-3_13"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.3034233"},{"key":"e_1_3_1_63_2","doi-asserted-by":"crossref","first-page":"7456","DOI":"10.1109\/ICCV48922.2021.00738","volume-title":"2021 IEEE\/CVF International Conference on Computer Vision (ICCV)","author":"Yuan H.","year":"2021","unstructured":"H. Yuan, D. Ni, and M. Wang. 2021. Spatio-temporal dynamic inference network for group activity recognition. In 2021 IEEE\/CVF International Conference on Computer Vision (ICCV). IEEE Computer Society, Los Alamitos, CA, 7456\u20137465. DOI: 10.1109\/ICCV48922.2021.00738"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2022.3193574"},{"key":"e_1_3_1_65_2","doi-asserted-by":"crossref","first-page":"1529","DOI":"10.1109\/ICCV.2015.179","volume-title":"2015 IEEE International Conference on Computer Vision (ICCV)","author":"Zheng S.","year":"2015","unstructured":"S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. S. Torr. 2015. Conditional random fields as recurrent neural networks. In 2015 IEEE International Conference on Computer Vision (ICCV). IEEE Computer Society, Los Alamitos, CA, 1529\u20131537. DOI: 10.1109\/ICCV.2015.179"},{"key":"e_1_3_1_66_2","first-page":"249","volume-title":"Proceedings of the 17th European Conference on Computer Vision (ECCV \u201922)","author":"Zhou Honglu","year":"2022","unstructured":"Honglu Zhou, Asim Kadav, Aviv Shamsian, Shijie Geng, Farley Lai, Long Zhao, Ting Liu, Mubbasir Kapadia, and Hans Peter Graf. 2022. COMPOSER: Compositional reasoning of group activity in videos with keypoint-only modality. In Proceedings of the 17th European Conference on Computer Vision (ECCV \u201922), 249\u2013266."}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3805798","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T13:19:25Z","timestamp":1778851165000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3805798"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,15]]},"references-count":65,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,5,31]]}},"alternative-id":["10.1145\/3805798"],"URL":"https:\/\/doi.org\/10.1145\/3805798","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,15]]},"assertion":[{"value":"2024-12-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-19","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-15","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}