{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T21:04:04Z","timestamp":1774991044535,"version":"3.50.1"},"reference-count":50,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2025,4,8]],"date-time":"2025-04-08T00:00:00Z","timestamp":1744070400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"publisher","award":["U21A20120"],"award-info":[{"award-number":["U21A20120"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"publisher","award":["52175033"],"award-info":[{"award-number":["52175033"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"publisher","award":["LQN25E050013"],"award-info":[{"award-number":["LQN25E050013"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"publisher","award":["6142606243106"],"award-info":[{"award-number":["6142606243106"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"publisher","award":["YQZX-B202401"],"award-info":[{"award-number":["YQZX-B202401"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"publisher","award":["XY2024012"],"award-info":[{"award-number":["XY2024012"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["U21A20120"],"award-info":[{"award-number":["U21A20120"]}]},{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["52175033"],"award-info":[{"award-number":["52175033"]}]},{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["LQN25E050013"],"award-info":[{"award-number":["LQN25E050013"]}]},{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["6142606243106"],"award-info":[{"award-number":["6142606243106"]}]},{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["YQZX-B202401"],"award-info":[{"award-number":["YQZX-B202401"]}]},{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["XY2024012"],"award-info":[{"award-number":["XY2024012"]}]},{"name":"National Key Laboratory of Transient Impact","award":["U21A20120"],"award-info":[{"award-number":["U21A20120"]}]},{"name":"National Key Laboratory of Transient Impact","award":["52175033"],"award-info":[{"award-number":["52175033"]}]},{"name":"National Key Laboratory of Transient Impact","award":["LQN25E050013"],"award-info":[{"award-number":["LQN25E050013"]}]},{"name":"National Key Laboratory of Transient Impact","award":["6142606243106"],"award-info":[{"award-number":["6142606243106"]}]},{"name":"National Key Laboratory of Transient Impact","award":["YQZX-B202401"],"award-info":[{"award-number":["YQZX-B202401"]}]},{"name":"National Key Laboratory of Transient Impact","award":["XY2024012"],"award-info":[{"award-number":["XY2024012"]}]},{"name":"Special Instrument Development and Cultivation Project of Zhejiang University","award":["U21A20120"],"award-info":[{"award-number":["U21A20120"]}]},{"name":"Special Instrument Development and Cultivation Project of Zhejiang University","award":["52175033"],"award-info":[{"award-number":["52175033"]}]},{"name":"Special Instrument Development and Cultivation Project of Zhejiang University","award":["LQN25E050013"],"award-info":[{"award-number":["LQN25E050013"]}]},{"name":"Special Instrument Development and Cultivation Project of Zhejiang University","award":["6142606243106"],"award-info":[{"award-number":["6142606243106"]}]},{"name":"Special Instrument Development and Cultivation Project of Zhejiang University","award":["YQZX-B202401"],"award-info":[{"award-number":["YQZX-B202401"]}]},{"name":"Special Instrument Development and Cultivation Project of Zhejiang University","award":["XY2024012"],"award-info":[{"award-number":["XY2024012"]}]},{"name":"A Project Supported by Scientific Research Fund of Zhejiang University","award":["U21A20120"],"award-info":[{"award-number":["U21A20120"]}]},{"name":"A Project Supported by Scientific Research Fund of Zhejiang University","award":["52175033"],"award-info":[{"award-number":["52175033"]}]},{"name":"A Project Supported by Scientific Research Fund of Zhejiang University","award":["LQN25E050013"],"award-info":[{"award-number":["LQN25E050013"]}]},{"name":"A Project Supported by Scientific Research Fund of Zhejiang University","award":["6142606243106"],"award-info":[{"award-number":["6142606243106"]}]},{"name":"A Project Supported by Scientific Research Fund of Zhejiang University","award":["YQZX-B202401"],"award-info":[{"award-number":["YQZX-B202401"]}]},{"name":"A Project Supported by Scientific Research Fund of Zhejiang University","award":["XY2024012"],"award-info":[{"award-number":["XY2024012"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Visual speech recognition is a technology that relies on visual information, offering unique advantages in noisy environments or when communicating with individuals with speech impairments. However, this technology still faces challenges, such as limited generalization ability due to different speech habits, high recognition error rates caused by confusable phonemes, and difficulties adapting to complex lighting conditions and facial occlusions. This paper proposes a lip reading data augmentation method\u2014Partition-Time Masking (PTM)\u2014to address these challenges and improve lip reading models\u2019 performance and generalization ability. Applying nonlinear transformations to the training data enhances the model\u2019s generalization ability when handling diverse speakers and environmental conditions. A lip-reading recognition model architecture, Swin Transformer and 3D Convolution (ST3D), was designed to overcome the limitations of traditional lip-reading models that use ResNet-based front-end feature extraction networks. By adopting a strategy that combines Swin Transformer and 3D convolution, the proposed model enhances performance. To validate the effectiveness of the Partition-Time Masking data augmentation method, experiments were conducted on the LRW video dataset using the DC-TCN model, achieving a peak accuracy of 92.15%. The ST3D model was validated on the LRW and LRW1000 video datasets, achieving a maximum accuracy of 56.1% on the LRW1000 dataset and 91.8% on the LRW dataset, outperforming current mainstream lip reading models and demonstrating superior performance on challenging easily confused samples.<\/jats:p>","DOI":"10.3390\/s25082366","type":"journal-article","created":{"date-parts":[[2025,4,9]],"date-time":"2025-04-09T03:32:53Z","timestamp":1744169573000},"page":"2366","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["A Novel Approach for Visual Speech Recognition Using the Partition-Time Masking and Swin Transformer 3D Convolutional Model"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6828-8563","authenticated-orcid":false,"given":"Xiangliang","family":"Zhang","sequence":"first","affiliation":[{"name":"The State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu","family":"Hu","sequence":"additional","affiliation":[{"name":"Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6522-6906","authenticated-orcid":false,"given":"Xiangzhi","family":"Liu","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1332-0746","authenticated-orcid":false,"given":"Yu","family":"Gu","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1735-9055","authenticated-orcid":false,"given":"Tong","family":"Li","sequence":"additional","affiliation":[{"name":"The Department of Sports Science, College of Education, Zhejiang University, Hangzhou 310027, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1278-671X","authenticated-orcid":false,"given":"Jibin","family":"Yin","sequence":"additional","affiliation":[{"name":"Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2797-0264","authenticated-orcid":false,"given":"Tao","family":"Liu","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,4,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1306","DOI":"10.1109\/JPROC.2003.817150","article-title":"Recent advances in the automatic recognition of audiovisual speech","volume":"91","author":"Potamianos","year":"2003","journal-title":"Proc. IEEE"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"656","DOI":"10.1097\/AUD.0b013e31812f7185","article-title":"Audiovisual integration and lipreading abilities of older adults with normal and impaired hearing","volume":"28","author":"Sommers","year":"2007","journal-title":"Ear Hear."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1080\/02699206.2020.1780473","article-title":"Speech rehabilitation in post-stroke aphasia using visual illustration of speech articulators: A case report study","volume":"35","author":"Haldin","year":"2021","journal-title":"Clin. Linguist. Phon."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2639","DOI":"10.1109\/TBME.2017.2654361","article-title":"Multimodal speech capture system for speech rehabilitation and learning","volume":"64","author":"Sebkhi","year":"2017","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_5","unstructured":"Zhou, M., Wang, Q., Li, Q., Jiang, P., Yang, J., Shen, C., Wang, C., and Ding, S. (2021). Securing face liveness detection using unforgeable lip motion patterns. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ma, X., Zhang, H., and Li, Y. (2017, January 24\u201326). Feature extraction method for lip-reading under variant lighting conditions. Proceedings of the 9th International Conference on Machine Learning and Computing, Singapore.","DOI":"10.1145\/3055635.3056576"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"5433","DOI":"10.1007\/s11760-024-03245-7","article-title":"A novel framework using 3D-CNN and BiLSTM model with dynamic learning rate scheduler for visual speech recognition","volume":"18","author":"Chandrabanshi","year":"2024","journal-title":"Signal Image Video Process."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Jeon, S., Elsharkawy, A., and Kim, M.S. (2021). Lipreading architecture based on multiple convolutional neural networks for sentence-level visual speech recognition. Sensors, 22.","DOI":"10.3390\/s22010072"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Noda, K., Yamaguchi, Y., Nakadai, K., Okuno, H.G., and Ogata, T. (2014, January 14\u201318). Lipreading using convolutional neural network. Proceedings of the Interspeech, Singapore.","DOI":"10.21437\/Interspeech.2014-293"},{"key":"ref_10","unstructured":"Weng, X., and Kitani, K. (2019). Learning spatio-temporal features with two-stream deep 3d cnns for lipreading. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yin, J., Qiao, Z., Han, L., and Zhang, X. (2025). EEG-based emotion recognition with autoencoder feature fusion and MSC-TimesNet model. Comput. Methods Biomech. Biomed. Eng., 1\u201318.","DOI":"10.1080\/10255842.2025.2477801"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1109\/34.982900","article-title":"Extraction of visual features for lipreading","volume":"24","author":"Matthews","year":"2002","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lee, B., Hasegawa-Johnson, M., Goudeseune, C., Kamdar, S., Borys, S., Liu, M., and Huang, T.S. (2004, January 4\u20138). AVICAR: Audio-visual speech corpus in a car environment. Proceedings of the Interspeech, Jeju Island, Republic of Korea.","DOI":"10.21437\/Interspeech.2004-424"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2421","DOI":"10.1121\/1.2229005","article-title":"An audio-visual corpus for speech perception and automatic speech recognition","volume":"120","author":"Cooke","year":"2006","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1254","DOI":"10.1109\/TMM.2009.2030637","article-title":"Lipreading with local spatiotemporal descriptors","volume":"11","author":"Zhao","year":"2009","journal-title":"IEEE Trans. Multimed."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Anina, I., Zhou, Z., Zhao, G., and Pietik\u00e4inen, M. (2015, January 4\u20138). Ouluvs2: A multi-view audiovisual database for non-rigid mouth motion analysis. Proceedings of the 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Ljubljana, Slovenia.","DOI":"10.1109\/FG.2015.7163155"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Son Chung, J., Senior, A., Vinyals, O., and Zisserman, A. (2017, January 21\u201326). Lip reading sentences in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.367"},{"key":"ref_18","unstructured":"Afouras, T., Chung, J.S., and Zisserman, A. (2018). LRS3-TED: A large-scale dataset for visual speech recognition. arXiv."},{"key":"ref_19","unstructured":"Chung, J.S., and Zisserman, A. (2016, January 20\u201324). Lip reading in the wild. Proceedings of the Computer Vision\u2013ACCV 2016: 13th Asian Conference on Computer Vision, Taipei, Taiwan. Revised Selected Papers, Part II 13."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Yang, S., Zhang, Y., Feng, D., Yang, M., Wang, C., Xiao, J., Long, K., Shan, S., and Chen, X. (2019, January 14\u201318). LRW-1000: A naturally-distributed large-scale benchmark for lip reading in the wild. Proceedings of the 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), Lille, France.","DOI":"10.1109\/FG.2019.8756582"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"107594","DOI":"10.1016\/j.patcog.2020.107594","article-title":"GridMix: Strong regularization through local context mapping","volume":"109","author":"Baek","year":"2021","journal-title":"Pattern Recognit."},{"key":"ref_22","unstructured":"Feng, D., Yang, S., Shan, S., and Chen, X. (2020). Learn an effective lip reading model without pains. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Yang, S., Xiao, J., Shan, S., and Chen, X. (2020, January 16\u201320). Can we read speech beyond the lips? rethinking roi selection for deep visual speech recognition. Proceedings of the 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), Buenos Aires, Argentina.","DOI":"10.1109\/FG47880.2020.00134"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/s11263-018-1097-z","article-title":"Facial landmark detection: A literature survey","volume":"127","author":"Wu","year":"2019","journal-title":"Int. J. Comput. Vis."},{"key":"ref_25","unstructured":"Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., and Yoo, Y. (November, January 27). Cutmix: Regularization strategy to train strong classifiers with localizable features. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.cviu.2018.10.003","article-title":"Pushing the boundaries of audiovisual word recognition using residual networks and LSTMs","volume":"176","author":"Stafylakis","year":"2018","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ma, P., Wang, Y., Petridis, S., Shen, J., and Pantic, M. (2022, January 7\u201313). Training strategies for improved lip-reading. Proceedings of the ICASSP 2022\u20132022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Virtual.","DOI":"10.1109\/ICASSP43922.2022.9746706"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Park, D.S., Chan, W., Zhang, Y., Chiu, C.C., Zoph, B., Cubuk, E.D., and Le, Q.V. (2019). Specaugment: A simple data augmentation method for automatic speech recognition. arXiv.","DOI":"10.21437\/Interspeech.2019-2680"},{"key":"ref_29","unstructured":"Petajan, E.D. (1984). Automatic Lipreading to Enhance Speech Recognition (Speech Reading), University of Illinois at Urbana-Champaign."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Petajan, E., Bischoff, B., Bodoff, D., and Brooke, N.M. (1988, January 15\u201319). An improved automatic lipreading system to enhance speech recognition. Proceedings of the SIGCHI conference on Human Factors in Computing Systems, Washington, DC, USA.","DOI":"10.1145\/57167.57170"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Goldschen, A.J., Garcia, O.N., and Petajan, E.D. (1997). Continuous automatic speech recognition by lipreading. Motion-Based Recognition, Springer.","DOI":"10.1007\/978-94-015-8935-2_14"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1214\/aoms\/1177697196","article-title":"A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains","volume":"41","author":"Baum","year":"1970","journal-title":"Ann. Math. Stat."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhao, G., Pietik\u00e4inen, M., and Hadid, A. (2007, January 28). Local spatiotemporal descriptors for visual recognition of spoken phrases. Proceedings of the International Workshop on Human-Centered Multimedia, Augsburg, Germany.","DOI":"10.1145\/1290128.1290138"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhou, Z., Zhao, G., and Pietik\u00e4inen, M. (2011, January 20\u201325). Towards a practical lipreading system. Proceedings of the CVPR 2011, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995345"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Pei, Y., Kim, T.K., and Zha, H. (2013, January 1\u20138). Unsupervised random forest manifold alignment for lipreading. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia.","DOI":"10.1109\/ICCV.2013.23"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1109\/34.927467","article-title":"Active appearance models","volume":"23","author":"Cootes","year":"2001","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_38","unstructured":"Assael, Y.M., Shillingford, B., Whiteson, S., and De Freitas, N. (2016). Lipnet: End-to-end sentence-level lipreading. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Tran, D., Bourdev, L., Fergus, R., Torresani, L., and Paluri, M. (2015, January 7\u201313). Learning spatiotemporal features with 3d convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.510"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11\u201317). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Martinez, B., Ma, P., Petridis, S., and Pantic, M. (2020, January 4\u20138). Lipreading using temporal convolutional networks. Proceedings of the ICASSP 2020\u20132020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9053841"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Ma, P., Wang, Y., Shen, J., Petridis, S., and Pantic, M. (2021, January 5\u20139). Lip-reading with densely connected temporal convolutional networks. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Virtual.","DOI":"10.1109\/WACV48630.2021.00290"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Hong, X., Yao, H., Wan, Y., and Chen, R. (2006, January 18\u201320). A PCA based visual DCT feature extraction method for lip-reading. Proceedings of the 2006 International Conference on Intelligent Information Hiding and Multimedia, Pasadena, CA, USA.","DOI":"10.1109\/IIH-MSP.2006.265008"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Ma, S., Wang, S., and Lin, X. (2020, January 27\u201329). A transformer-based model for sentence-level Chinese Mandarin lipreading. Proceedings of the 2020 IEEE Fifth International Conference on Data Science in Cyberspace (DSC), Hong Kong, China.","DOI":"10.1109\/DSC50466.2020.00020"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"77205","DOI":"10.1109\/ACCESS.2022.3193231","article-title":"A lip reading method based on 3D convolutional vision transformer","volume":"10","author":"Wang","year":"2022","journal-title":"IEEE Access"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"4507","DOI":"10.1109\/TCSVT.2023.3282224","article-title":"Lipformer: Learning to lipread unseen speakers based on visual-landmark transformers","volume":"33","author":"Xue","year":"2023","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Koumparoulis, A., and Potamianos, G. (2022, January 22\u201327). Accurate and resource-efficient lipreading with efficientnetv2 and transformers. Proceedings of the ICASSP 2022\u20132022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.","DOI":"10.1109\/ICASSP43922.2022.9747729"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Ma, P., Martinez, B., Petridis, S., and Pantic, M. (2021, January 6\u201311). Towards practical lipreading with distilled and efficient models. Proceedings of the ICASSP 2021\u20132021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.","DOI":"10.1109\/ICASSP39728.2021.9415063"},{"key":"ref_50","unstructured":"Kim, M., Yeo, J.H., and Ro, Y.M. (March, January 22). Distinguishing homophenes using multi-head visual-audio memory for lip reading. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/8\/2366\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:12:49Z","timestamp":1760029969000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/8\/2366"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,8]]},"references-count":50,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2025,4]]}},"alternative-id":["s25082366"],"URL":"https:\/\/doi.org\/10.3390\/s25082366","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,8]]}}}