{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,24]],"date-time":"2026-07-24T17:28:38Z","timestamp":1784914118544,"version":"3.55.0"},"reference-count":45,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,3,23]],"date-time":"2023-03-23T00:00:00Z","timestamp":1679529600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>Modern action recognition techniques frequently employ two networks: the spatial stream, which accepts input from RGB frames, and the temporal stream, which accepts input from optical flow. Recent researches use 3D convolutional neural networks that employ spatiotemporal filters on both streams. Although mixing flow with RGB enhances performance, correct optical flow computation is expensive and adds delay to action recognition. In this study, we present a method for training a 3D CNN using RGB frames that replicates the motion stream and, as a result, does not require flow calculation during testing. To begin, in contrast to the SE block, we suggest a channel excitation module (CE module). Experiments have shown that the CE module can improve the feature extraction capabilities of a 3D network and that the effect is superior to the SE block. Second, for action recognition training, we adopt a linear mix of loss based on knowledge distillation and standard cross-entropy loss to effectively leverage appearance and motion information. The Intensified Motion RGB Stream is the stream trained with this combined loss (IMRS). IMRS surpasses RGB or Flow as a single stream; for example, HMDB51 achieves 73.5% accuracy, while RGB and Flow streams score 65.6% and 69.1% accuracy, respectively. Extensive experiments confirm the effectiveness of our proposed method. The comparison with other models proves that our model has good competitiveness in behavior recognition.<\/jats:p>","DOI":"10.3389\/fnbot.2023.1050167","type":"journal-article","created":{"date-parts":[[2023,3,23]],"date-time":"2023-03-23T07:28:42Z","timestamp":1679556522000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["3D network with channel excitation and knowledge distillation for action recognition"],"prefix":"10.3389","volume":"17","author":[{"given":"Zhengping","family":"Hu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianzeng","family":"Mao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianxin","family":"Yao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuai","family":"Bi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,3,23]]},"reference":[{"key":"B1","first-page":"4724","article-title":"\u201cQuo vadis, action recognition? A new model and the kinetics dataset,\u201d","volume-title":"2017 IEEE Computer Vision and Pattern Recognition","author":"Carreira","year":"2017"},{"key":"B2","first-page":"4511","article-title":"\u201cTwo-stream video classification with cross-modality attention,\u201d","volume-title":"IEEE International Conference on Computer Vision Workshop","author":"Chi","year":"2019"},{"key":"B3","first-page":"7874","article-title":"\u201cMARS: motion-augmented RGB stream for action recognition,\u201d","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Crasto","year":"2019"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2021.3055233.","article-title":"Forecasting action through contact representations from first person video","author":"Dessalene","year":"2021","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence PP"},{"key":"B5","first-page":"299","article-title":"\u201cSpatio-temporal channel correlation networks for action classification,\u201d","volume-title":"European Conference on Computer Vision","author":"Diba","year":"2018"},{"key":"B6","unstructured":"Temporal 3D ConvNets: new architecture and transfer learning for video classification\n            DibaA.\n            FayyazM.\n            SharmaV.\n            KaramiA. H.\n            ArzaniM. M.\n            YousefzadehR.\n          arXiv Preprint.2017"},{"key":"B7","first-page":"2625","article-title":"\u201cLong-term recurrent convolutional networks for visual recognition and description,\u201d","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition","author":"Donahue","year":"2015"},{"key":"B8","first-page":"4729","article-title":"\u201c3D CNNs with adaptive temporal feature resolutions,\u201d","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Fayyaz","year":"2021"},{"key":"B9","first-page":"200","article-title":"\u201cX3D: expanding architectures for efficient video recognition,\u201d","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Feichtenhofer","year":"2020"},{"key":"B10","first-page":"6201","article-title":"\u201cSlowFast networks for video recognition,\u201d","volume-title":"IEEE\/CVF International Conference on Computer Vision","author":"Feichtenhofer","year":"2019"},{"key":"B11","first-page":"3468","article-title":"\u201cSpatiotemporal residual networks for video action recognition,\u201d","volume-title":"Neural Information Processing Systems","author":"Feichtenhofer","year":"2016"},{"key":"B12","first-page":"3141","article-title":"\u201cDual attention network for scene segmentation,\u201d","volume-title":"2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Fu","year":"2019"},{"key":"B13","first-page":"6546","article-title":"\u201cCan spatiotemporal 3D CNNs retrace the history of 2D CNNs and ImageNet,\u201d","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Hara","year":"2018"},{"key":"B14","unstructured":"Distilling the knowledge in a neural network\n            HintonG. E.\n            VinyalsO.\n            DeanJ.\n          Comput. Sci.2015"},{"key":"B15","doi-asserted-by":"crossref","first-page":"2011","DOI":"10.1109\/CVPR.2018.00745","article-title":"\u201cSqueeze-and-excitation networks,\u201d","volume-title":"2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Hu","year":"2018"},{"key":"B16","doi-asserted-by":"publisher","first-page":"45753","DOI":"10.1109\/ACCESS.2020.2978223","article-title":"Efficient parallel inflated 3D convolution architecture for action recognition","volume":"8","author":"Huang","year":"2020","journal-title":"IEEE Access"},{"key":"B17","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1109\/TPAMI.2012.59","article-title":"3D Convolutional neural networks for human action recognition","volume":"35","author":"Ji","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"B18","first-page":"2000","article-title":"\u201cSTM: spatiotemporal and motion encoding for action recognition,\u201d","volume-title":"IEEE\/CVF International Conference on Computer Vision","author":"Jiang","year":"2019"},{"key":"B19","article-title":"Human centric spatio-temporal action localization.","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition","author":"Jiang","year":"2018"},{"key":"B20","first-page":"1725","article-title":"\u201cLarge-scale video classification with convolutional neural networks,\u201d","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition","author":"Karpathy","year":"2014"},{"key":"B21","first-page":"2556","article-title":"\u201cHMDB: a large video database for human motion recognition,\u201d","volume-title":"International Conference on Computer Vision","author":"Kuehne","year":"2011"},{"key":"B22","first-page":"392","article-title":"\u201cMotion feature network: fixed motion filter for action recognition,\u201d","volume-title":"European Conference on Computer Vision","author":"Lee","year":"2018"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.patcog.2019.107037","article-title":"Spatio-temporal deformable 3D ConvNets with attention for action recognition","volume":"98","author":"Li","year":"2020","journal-title":"Pattern Recogn."},{"key":"B24","article-title":"CT-Net: channel tensorization network for video classification","author":"Li","year":"2021","journal-title":"arXiv Preprint."},{"key":"B25","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR42600.2020.00099","article-title":"\u201cTEA: temporal excitation and aggregation for action recognition,\u201d","volume-title":"IEEE Conference of Computer Vision and Pattern Recognition","author":"Li","year":"2020"},{"key":"B26","first-page":"7082","article-title":"\u201cTSM: temporal shift module for efficient video understanding,\u201d","volume-title":"2019 IEEE\/CVF International Conference on Computer Vision","author":"Lin","year":"2019"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2022.3148588","article-title":"Attention-driven appearance-motion fusion network for action recognition","author":"Liu","year":"2022","journal-title":"IEEE Trans. Multimedia"},{"key":"B28","first-page":"5650","article-title":"\u201cAsynchronous temporal fields for action recognition,\u201d","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition","author":"Sigurdsson","year":"2017"},{"key":"B29","first-page":"568","article-title":"\u201cTwo-stream convolutional networks for action recognition in videos,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Simonyan","year":"2014"},{"key":"B30","unstructured":"UCF101: a dataset of 101 human actions classes from videos in the wild\n            SoomroK.\n            ZamirA. R.\n            ShahM.\n          arXiv Preprint.2012"},{"key":"B31","first-page":"4489","article-title":"\u201cLearning spatiotemporal features with 3D convolutional networks,\u201d","volume-title":"IEEE International Conference on Computer Vision","author":"Tran","year":"2015"},{"key":"B32","first-page":"6450","article-title":"\u201cA closer look at spatiotemporal convolutions for action recognition,\u201d","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Tran","year":"2018"},{"key":"B33","first-page":"3551","article-title":"\u201cAction recognition with improved trajectories,\u201d","volume-title":"IEEE International Conference on Computer Vision","author":"Wang","year":"2013"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46484-8_2","article-title":"Temporal segment networks: towards good practices for deep action recognition","author":"Wang","year":"","journal-title":"arXiv Preprint."},{"key":"B35","first-page":"20","author":"Wang","year":""},{"key":"B36","article-title":"\u201cCBAM: convolutional block attention module,\u201d","author":"Woo","year":"2018","journal-title":"European Conference on Computer Vision"},{"key":"B37","first-page":"305","article-title":"\u201cRethinking spatiotemporal feature learning: speed-accuracy trade-offs in video classification,\u201d","volume-title":"European Conference on Computer Vision","author":"Xie","year":"2018"},{"key":"B38","first-page":"10153","article-title":"\u201cG-TAD: sub-graph localization for temporal action detection,\u201d","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Xu","year":"2020"},{"key":"B39","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v29i1.9671","article-title":"\u201cRobot learning manipulation action plans by \u201cwatching\u201c unconstrained videos from the world wide web,\u201d","author":"Yang","year":"2015","journal-title":"AAAI Conference on Artificial Intelligence"},{"key":"B40","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-540-74936-3_22","article-title":"\u201cA duality based approach for realtime TV-L1 optical flow,\u201d","volume-title":"DAGM Conference on Pattern Recognition","author":"Zach","year":"2007"},{"key":"B41","article-title":"\u201cVisualizing and understanding convolutional networks,\u201d","author":"Zeiler","year":"2014","journal-title":"European Conference on Computer Vision"},{"key":"B42","article-title":"\u201cBottom-up temporal action localization with mutual regularization,\u201d","author":"Zhao","year":"2020","journal-title":"European Conference on Computer Vision"},{"key":"B43","unstructured":"A3D: adaptive 3D networks for video action recognition\n            ZhuS.\n            YangT.\n            MendietaM.\n            ChenC.\n          arXiv Preprint.2020"},{"key":"B44","unstructured":"The kinetics human action video dataset\n            ZissermanA.\n            CarreiraJ.\n            SimonyanK.\n            KayW.\n            ZhangB. H.\n            HillierC.\n          arXiv Preprint.2017"},{"key":"B45","first-page":"695","article-title":"\u201cECO: efficient convolutional network for online video understanding,\u201d","volume-title":"European Conference on Computer Vision","author":"Zolfaghari","year":"2018"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1050167\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,23]],"date-time":"2023-03-23T07:29:16Z","timestamp":1679556556000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1050167\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,23]]},"references-count":45,"alternative-id":["10.3389\/fnbot.2023.1050167"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1050167","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,23]]},"article-number":"1050167"}}