{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T02:42:47Z","timestamp":1774579367772,"version":"3.50.1"},"reference-count":43,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,2,12]],"date-time":"2025-02-12T00:00:00Z","timestamp":1739318400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>Human skeleton-based action recognition is an important task in the field of computer vision. In recent years, masked autoencoder (MAE) has been used in various fields due to its powerful self-supervised learning ability and has achieved good results in masked data reconstruction tasks. However, in visual classification tasks such as action recognition, the limited ability of the encoder to learn features in the autoencoder structure results in poor classification performance. We propose to enhance the encoder's feature extraction ability in classification tasks by leveraging the latent space of variational autoencoder (VAE) and further replace it with the latent space of vector quantized variational autoencoder (VQVAE). The constructed models are called SkeletonMVAE and SkeletonMVQVAE, respectively. In SkeletonMVAE, we constrain the latent variables to represent features in the form of distributions. In SkeletonMVQVAE, we discretize the latent variables. These help the encoder learn deeper data structures and more discriminative and generalized feature representations. The experiment results on the NTU-60 and NTU-120 datasets demonstrate that our proposed method can effectively improve the classification accuracy of the encoder in classification tasks and its generalization ability in the case of few labeled data. SkeletonMVAE exhibits stronger classification ability, while SkeletonMVQVAE exhibits stronger generalization in situations with fewer labeled data.<\/jats:p>","DOI":"10.3389\/fnbot.2025.1482281","type":"journal-article","created":{"date-parts":[[2025,3,18]],"date-time":"2025-03-18T08:23:57Z","timestamp":1742286237000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Latent space improved masked reconstruction model for human skeleton-based action recognition"],"prefix":"10.3389","volume":"19","author":[{"given":"Enqing","family":"Chen","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xueting","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xin","family":"Guo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ying","family":"Zhu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dong","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,2,12]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"2206","DOI":"10.1109\/TCSVT.2020.3019293","article-title":"Fuzzy integral-based CNN classifier fusion for 3D skeleton action recognition","volume":"31","author":"Banerjee","year":"2020","journal-title":"IEEE Trans. Circ. Syst. Video Technol"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.675","article-title":"\u201cSST: single-stream temporal action proposals,\u201d","author":"Buch","year":"2017","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.143","author":"Cao","year":"2017","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01311","article-title":"\u201cChannel-wise topology refinement graph convolution for skeleton-based action recognition,\u201d","author":"Chen","year":"2021","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision"},{"key":"B5","first-page":"185","article-title":"\u201cHierarchically self-supervised transformer for human skeleton representation learning,\u201d","volume-title":"European Conference on Computer Vision","author":"Chen","year":"2022"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1420","DOI":"10.1109\/TETCI.2023.3298535","article-title":"A generative model for surrogates of spatial-temporal wildfire nowcasting","volume":"7","author":"Cheng","year":"2023","journal-title":"IEEE Trans. Emerg. Topics Comput. Intell"},{"key":"B7","article-title":"Bert: pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2018","journal-title":"arXiv preprint arXiv:1810.04805"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i1.25127","article-title":"\u201cHierarchical contrast for unsupervised skeleton-based action representation learning,\u201d","author":"Dong","year":"2023","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298714","article-title":"\u201cHierarchical recurrent neural network for skeleton based action recognition,\u201d","author":"Du","year":"2015","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00298","article-title":"\u201cRevisiting skeleton-based action recognition,\u201d","author":"Duan","year":"2022","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.256","article-title":"\u201cRMPE: regional multi-person pose estimation,\u201d","author":"Fang","year":"2017","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00630","article-title":"\u201cSlowfast networks for video recognition,\u201d","author":"Feichtenhofer","year":"2019","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i1.19957","article-title":"\u201cContrastive learning from extremely augmented skeleton sequences for self-supervised action recognition,\u201d","author":"Guo","year":"2022","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01553","article-title":"\u201cMasked autoencoders are scalable vision learners,\u201d","author":"He","year":"2022","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B15","doi-asserted-by":"publisher","first-page":"807","DOI":"10.1109\/TCSVT.2016.2628339","article-title":"Skeleton optical spectra-based action recognition using convolutional neural networks","volume":"28","author":"Hou","year":"2016","journal-title":"IEEE Trans. Circ. Syst. Video Technol"},{"key":"B16","doi-asserted-by":"crossref","first-page":"415","DOI":"10.23919\/APSIPAASC55919.2022.9980239","article-title":"\u201cUsing prosodic phrase-based VQVAE on audio albert for speech emotion recognition,\u201d","volume-title":"2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)","author":"Hsu","year":"2022"},{"key":"B17","article-title":"Part aware contrastive learning for self-supervised action recognition","author":"Hua","year":"2023","journal-title":"arXiv preprint arXiv:2305.00666"},{"key":"B18","article-title":"Auto-encoding variational bayes","author":"Kingma","year":"2013","journal-title":"arXiv preprint arXiv:1312.6114"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00471","article-title":"\u201c3D human action representation learning via cross-view consistency pursuit,\u201d","author":"Li","year":"2021","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00572","article-title":"Deep independently recurrent neural network (INDRNN)","author":"Li","year":"2019","journal-title":"arXiv preprint arXiv:1910.06251"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00234","article-title":"\u201cActionLet-dependent contrastive learning for unsupervised skeleton-based action recognition,\u201d","author":"Lin","year":"2023","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2020-1520","article-title":"\u201cTemporal attention convolutional network for speech emotion recognition with latent representation,\u201d","author":"Liu","year":"2020","journal-title":"INTERSPEECH"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00022","article-title":"\u201cDisentangling and unifying graph convolutions for skeleton-based action recognition,\u201d","author":"Liu","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B24","article-title":"RTMO: towards high-performance one-stage real-time multi-person pose estimation","author":"Lu","year":"2023","journal-title":"arXiv preprint arXiv:2312.07526"},{"key":"B25","article-title":"\u201cPytorch: an imperative style, high-performance deep learning library,\u201d","author":"Paszke","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B26","doi-asserted-by":"publisher","first-page":"218","DOI":"10.1109\/TMM.2023.3263288","article-title":"MAR: Masked autoencoders for efficient action recognition","volume":"26","author":"Qing","year":"2023","journal-title":"IEEE Trans. Multim"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.115","article-title":"\u201cNTU RGB+ D: a large scale dataset for 3D human activity analysis,\u201d","author":"Shahroudy","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B28","article-title":"\u201cTwo-stream convolutional networks for action recognition in videos,\u201d","author":"Simonyan","year":"2014","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1145\/3474085.3475307","article-title":"\u201cSkeleton-contrastive 3D action representation learning,\u201d","author":"Thoker","year":"2021","journal-title":"Proceedings of the 29th ACM International Conference on Multimedia"},{"key":"B30","first-page":"10078","article-title":"\u201cVideomae: masked autoencoders are data-efficient learners for self-supervised video pre-training,\u201d","author":"Tong","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B31","article-title":"\u201cNeural discrete representation learning,\u201d","author":"Van Den Oord","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B32","doi-asserted-by":"publisher","first-page":"1510","DOI":"10.1109\/TPAMI.2017.2712608","article-title":"Long-term temporal convolutions for action recognition","volume":"40","author":"Varol","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B33","doi-asserted-by":"crossref","first-page":"1080","DOI":"10.1109\/ICIP49359.2023.10222596","article-title":"\u201cUnsupervised anomaly detection with local-sensitive VQVAE and global-sensitive transformers,\u201d","volume-title":"2023 IEEE International Conference on Image Processing (ICIP)","author":"Wang","year":"2023"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1145\/2964284.2967191","article-title":"\u201cAction recognition based on joint trajectory maps using convolutional neural networks,\u201d","author":"Wang","year":"2016","journal-title":"Proceedings of the 24th ACM International Conference on Multimedia"},{"key":"B35","article-title":"IIP-transformer: Intra-inter-part transformer for skeleton-based action recognition","author":"Wang","year":"2021","journal-title":"arXiv preprint arXiv:2110.13385"},{"key":"B36","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1109\/ICMEW59549.2023.00045","article-title":"\u201cSkeletonmae: spatial-temporal masked autoencoders for self-supervised skeleton action recognition,\u201d","volume-title":"2023 IEEE International Conference on Multimedia and Expo Workshops (ICMEW)","author":"Wu","year":"2023"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00098","article-title":"\u201cDeep kinematics analysis for monocular 3D human pose estimation,\u201d","author":"Xu","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B38","doi-asserted-by":"publisher","first-page":"12266","DOI":"10.1007\/s11227-023-05139-w","article-title":"CNN-VAE: an intelligent text representation algorithm","volume":"79","author":"Xu","year":"2023","journal-title":"J. Supercomput"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.12328","article-title":"\u201cSpatial temporal graph convolutional networks for skeleton-based action recognition,\u201d","author":"Yan","year":"2018","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"B40","article-title":"Variational autoencoder learns better feature representations for EEG-based obesity classification","author":"Yue","year":"2023","journal-title":"arXiv preprint arXiv:2302.00789"},{"key":"B41","first-page":"36","article-title":"\u201cContrastive positive mining for unsupervised 3d action representation learning,\u201d","volume-title":"European Conference on Computer Vision","author":"Zhang","year":"2022"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1145\/3474085.3475473","article-title":"\u201cSTST: spatial-temporal specialized transformer for skeleton-based action recognition,\u201d","author":"Zhang","year":"2021","journal-title":"Proceedings of the 29th ACM International Conference on Multimedia"},{"key":"B43","doi-asserted-by":"publisher","first-page":"15744","DOI":"10.1039\/D2CP05975D","article-title":"Analyzing drop coalescence in microfluidic devices with a deep learning generative model","volume":"25","author":"Zhu","year":"2023","journal-title":"Phys. Chem. Chem. Phys"}],"updated-by":[{"DOI":"10.3389\/fnbot.2025.1587250","type":"erratum","label":"Erratum","source":"publisher","updated":{"date-parts":[[2025,3,18]],"date-time":"2025-03-18T00:00:00Z","timestamp":1742256000000}}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2025.1482281\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,18]],"date-time":"2025-03-18T08:24:03Z","timestamp":1742286243000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2025.1482281\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,12]]},"references-count":43,"aliases":["10.3389\/fnbot.2024.1482281"],"alternative-id":["10.3389\/fnbot.2025.1482281"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2025.1482281","relation":{"erratum":[{"id-type":"doi","id":"10.3389\/fnbot.2025.1587250","asserted-by":"object"}]},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,12]]},"article-number":"1482281"}}