{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T11:27:36Z","timestamp":1786966056513,"version":"build-2736575974"},"reference-count":41,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2026,8,16]],"date-time":"2026-08-16T00:00:00Z","timestamp":1786838400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Accessibility-oriented sign language recognition plays an important role in intelligent human\u2013computer interaction, but cross-subject recognition remains challenging because test signers are unseen during training. Variations in appearance, motion style, and modality quality among different signers introduce significant distribution differences between training and testing samples, limiting model generalization. Although multimodal approaches commonly assume that RGB, skeleton, and depth modalities provide complementary information, their effectiveness under cross-subject recognition remains insufficiently explored. This study investigates modality effectiveness in SLR500 word-level sign language recognition under a cross-subject protocol. A unified evaluation framework is established to compare RGB-only, skeleton-only, RGB\u2013skeleton fusion, and RGB\u2013skeleton\u2013depth fusion under consistent data splits, preprocessing procedures, training settings, and evaluation metrics. Sample-level prediction transitions are further analyzed to examine how fusion changes recognition outcomes relative to the skeleton-based reference model. The results show that the skeleton modality provides more reliable recognition performance under the evaluated cross-subject setting, while the tested fusion strategies do not consistently improve over the skeleton-only configuration. Based on the modality-effectiveness analysis, a skeleton-based reference model is established using a Spatial\u2013Temporal Graph Convolutional Network (ST-GCN)-based spatio-temporal encoder, learned confidence-weighted temporal pooling, and a lightweight classifier. Multi-seed experiments further verify the stability of the model, which achieves 86.37% Top-1 accuracy and 96.86% Top-5 accuracy on the SLR500 cross-subject test set. These results emphasize the importance of evaluating modality reliability before designing complex fusion strategies for cross-subject sign language recognition.<\/jats:p>","DOI":"10.3390\/s26165184","type":"journal-article","created":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T11:00:05Z","timestamp":1786964405000},"page":"5184","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Modality Effectiveness in Cross-Subject Sign Language Recognition: A Skeleton Perspective"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-6499-1328","authenticated-orcid":false,"given":"Sijie","family":"Miao","sequence":"first","affiliation":[{"name":"School of Computer and Software Engineering, Xihua University, Chengdu 610039, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-1472-4077","authenticated-orcid":false,"given":"Jiaxin","family":"He","sequence":"additional","affiliation":[{"name":"School of Computer and Software Engineering, Xihua University, Chengdu 610039, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6162-1295","authenticated-orcid":false,"given":"Lu","family":"Zeng","sequence":"additional","affiliation":[{"name":"School of Computer and Software Engineering, Xihua University, Chengdu 610039, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,16]]},"reference":[{"key":"ref_1","unstructured":"Koller, O. (2020). Quantitative Survey of the State of the Art in Sign Language Recognition. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Bragg, D., Koller, O., Bellard, M., Berke, L., Boudreault, P., Braffort, A., Caselli, N., Huenerfauth, M., Kacorri, H., and Verhoef, T. (2019, January 28\u201330). Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective. Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS\u201919, Pittsburgh, PA, USA.","DOI":"10.1145\/3308561.3353774"},{"key":"ref_3","unstructured":"Camgoz, N.C., Koller, O., Hadfield, S., and Bowden, R. (2020, January 13\u201319). Sign Language Transformers: Joint End-to-End Sign Language Recognition and Translation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Li, D., Rodriguez, C., Yu, X., and Li, H. (2020, January 1\u20135). Word-Level Deep Sign Language Recognition from Video: A New Large-Scale Dataset and Methods Comparison. Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO, USA.","DOI":"10.1109\/WACV45572.2020.9093512"},{"key":"ref_5","unstructured":"Joze, H.R.V., and Koller, O. (2018). MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Language. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"181340","DOI":"10.1109\/ACCESS.2020.3028072","article-title":"AUTSL: A Large Scale Multi-Modal Turkish Sign Language Dataset and Baseline Methods","volume":"8","author":"Sincan","year":"2020","journal-title":"IEEE Access"},{"key":"ref_7","unstructured":"Simonyan, K., and Zisserman, A. (2014, January 8\u201313). Two-Stream Convolutional Networks for Action Recognition in Videos. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Montreal, QC, Canada."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Tran, D., Bourdev, L., Fergus, R., Torresani, L., and Paluri, M. (2015, January 7\u201313). Learning Spatiotemporal Features with 3D Convolutional Networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.510"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Carreira, J., and Zisserman, A. (2017, January 21\u201326). Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.502"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hara, K., Kataoka, H., and Satoh, Y. (2018, January 18\u201322). Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00685"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., and Van Gool, L. (2016). Temporal Segment Networks: Towards Good Practices for Deep Action Recognition. Computer Vision\u2013ECCV 2016, Springer.","DOI":"10.1007\/978-3-319-46484-8_2"},{"key":"ref_13","unstructured":"Feichtenhofer, C., Fan, H., Malik, J., and He, K. (November, January 27). SlowFast Networks for Video Recognition. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea."},{"key":"ref_14","unstructured":"Sun, C., Myers, A., Vondrick, C., Murphy, K., and Schmid, C. (November, January 27). VideoBERT: A Joint Model for Video and Language Representation Learning. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wang, X., Girshick, R., Gupta, A., and He, K. (2018, January 18\u201322). Non-Local Neural Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00813"},{"key":"ref_16","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4\u20139). Attention Is All You Need. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Long Beach, CA, USA."},{"key":"ref_17","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021, January 3\u20137). An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale. Proceedings of the International Conference on Learning Representations (ICLR), Virtual."},{"key":"ref_18","unstructured":"Bertasius, G., Wang, H., and Torresani, L. (2021, January 18\u201324). Is Space-Time Attention All You Need for Video Understanding?. Proceedings of the International Conference on Machine Learning (ICML), Virtual."},{"key":"ref_19","unstructured":"Defferrard, M., Bresson, X., and Vandergheynst, P. (2016, January 5\u201310). Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Barcelona, Spain."},{"key":"ref_20","unstructured":"Kipf, T.N., and Welling, M. (2017, January 24\u201326). Semi-Supervised Classification with Graph Convolutional Networks. Proceedings of the International Conference on Learning Representations (ICLR), Toulon, France."},{"key":"ref_21","unstructured":"Veli\u010dkovi\u0107, P., Cucurull, G., Casanova, A., Romero, A., Li\u00f2, P., and Bengio, Y. (May, January 30). Graph Attention Networks. Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Yan, S., Xiong, Y., and Lin, D. (2018, January 2\u20137). Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.12328"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Shi, L., Zhang, Y., Cheng, J., and Lu, H. (2019, January 16\u201320). Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01230"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Cheng, K., Zhang, Y., He, X., Chen, W., Cheng, J., and Lu, H. (2020, January 13\u201319). Skeleton-Based Action Recognition with Shift Graph Convolutional Network. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00026"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Chen, Y., Zhang, Z., Yuan, C., Li, B., Deng, Y., and Hu, W. (2021, January 10\u201317). Channel-Wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.01311"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Jiang, S., Sun, B., Wang, L., Bai, Y., Li, K., and Fu, Y. (2021, January 19\u201325). Skeleton Aware Multi-Modal Sign Language Recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, TN, USA.","DOI":"10.1109\/CVPRW53098.2021.00380"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Hu, H., Zhao, W., Zhou, W., Wang, Y., and Li, H. (2021, January 10\u201317). SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.01090"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"103219","DOI":"10.1016\/j.cviu.2021.103219","article-title":"Skeleton-Based Action Recognition via Spatial and Temporal Transformer Networks","volume":"208\u2013209","author":"Plizzari","year":"2021","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"768","DOI":"10.1109\/TMM.2021.3059098","article-title":"Spatial-Temporal Multi-Cue Network for Sign Language Recognition and Translation","volume":"24","author":"Zhou","year":"2022","journal-title":"IEEE Trans. Multimed."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Huang, J., Zhou, W., Zhang, Q., Li, H., and Li, W. (2018, January 2\u20137). Video-Based Sign Language Recognition without Temporal Segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11903"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Pu, J., Zhou, W., and Li, H. (2019, January 15\u201320). Iterative Alignment Network for Continuous Sign Language Recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00429"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Pu, J., Zhou, W., Hu, H., and Li, H. (2020, January 12\u201316). Boosting Continuous Sign Language Recognition via Cross Modality Augmentation. Proceedings of the 28th ACM International Conference on Multimedia (ACM MM), Seattle, WA, USA.","DOI":"10.1145\/3394171.3413931"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1109\/TPAMI.2018.2798607","article-title":"Multimodal Machine Learning: A Survey and Taxonomy","volume":"41","author":"Ahuja","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","unstructured":"Tsai, Y.H.H., Bai, S., Liang, P.P., Kolter, J.Z., Morency, L.P., and Salakhutdinov, R. (August, January 28). Multimodal Transformer for Unaligned Multimodal Language Sequences. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), Florence, Italy."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"3652","DOI":"10.1109\/TMM.2026.3654340","article-title":"Learning Dual Modality Interactions for Event-Based Motion Deblurring","volume":"28","author":"Xiao","year":"2026","journal-title":"IEEE Trans. Multimed."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"3270","DOI":"10.1109\/LSP.2025.3599733","article-title":"L3FMamba: Low-Light Light Field Image Enhancement with Prior-Injected State Space Models","volume":"32","author":"Liu","year":"2025","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_37","unstructured":"Rosenstein, M.T., Marx, Z., Kaelbling, L.P., and Dietterich, T.G. (2005, January 5\u20138). To Transfer or Not to Transfer. Proceedings of the NIPS Workshop on Transfer Learning, Vancouver, BC, Canada."},{"key":"ref_38","unstructured":"Ruder, S. (2017). An Overview of Multi-Task Learning in Deep Neural Networks. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_40","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 6\u201311). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Proceedings of the International Conference on Machine Learning (ICML), Lille, France. Available online: https:\/\/proceedings.mlr.press\/v37\/ioffe15.html."},{"key":"ref_41","first-page":"1929","article-title":"Dropout: A Simple Way to Prevent Neural Networks from Overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/16\/5184\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T11:06:30Z","timestamp":1786964790000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/16\/5184"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,16]]},"references-count":41,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2026,8]]}},"alternative-id":["s26165184"],"URL":"https:\/\/doi.org\/10.3390\/s26165184","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,16]]}}}