{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,18]],"date-time":"2025-10-18T00:12:09Z","timestamp":1760746329380,"version":"build-2065373602"},"reference-count":45,"publisher":"World Scientific Pub Co Pte Ltd","issue":"02","funder":[{"name":"the China Scholarship Council","award":["202108515074"],"award-info":[{"award-number":["202108515074"]}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"publisher","award":["J2023-070"],"award-info":[{"award-number":["J2023-070"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]},{"name":"the independent research project of the Civil Aviation Flight Technology and Flight Safety Key Laboratory of CAFUC","award":["FZ2022ZZZ06"],"award-info":[{"award-number":["FZ2022ZZZ06"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J CIRCUIT SYST COMP"],"published-print":{"date-parts":[[2026,1,30]]},"abstract":"<jats:p> In the field of bridging communication gaps between deaf and hearing communities, sign language recognition (SLR) and sign language translation (SLT) systems face persistent challenges in cross-lingual contexts as the understanding ability of unlabeled data is limited, the fusion depth of traditional fusion modes is insufficient, and inadequate temporal modeling, which hinder practical performance. To address these limitations, this study proposes TriFuseNet, an innovative open-vocabulary tri-modal fusion framework grounded in intersemiotic translation theory, dynamic equivalence theory, and multimodal discourse analysis, which integrates RGB images, textual semantics and skeletal keypoints through a hierarchical interaction mechanism to achieve fine-grained cross-modal alignment. The framework introduces a vision-language interaction module to bridge visual features with linguistic structures, while a bilinear state-space model (BSSM) captures long-range temporal dynamics for gesture-linguistic coherence. Additionally, region-specific attention mechanisms prioritize critical body regions to model culturally specific gestures with nuanced spatial-temporal details. Experimental evaluations on the CSL-Daily and PHOENIX-14T datasets demonstrate TriFuseNet\u2019s superiority over state-of-the-art methods, achieving 2.3% and 1.6% reductions in word error rate (WER), respectively. <\/jats:p>","DOI":"10.1142\/s021812662550375x","type":"journal-article","created":{"date-parts":[[2025,6,29]],"date-time":"2025-06-29T23:47:43Z","timestamp":1751240863000},"source":"Crossref","is-referenced-by-count":0,"title":["TriFuseNet: Open-Vocabulary Sign Language Translation via Hierarchical Image-Text-Keypoint Fusion"],"prefix":"10.1142","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-2401-3353","authenticated-orcid":false,"given":"Ai","family":"Wang","sequence":"first","affiliation":[{"name":"School of Foreign Languages, Civil Aviation Flight University of China, Guanghan, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-5681-2273","authenticated-orcid":false,"given":"Junhui","family":"Li","sequence":"additional","affiliation":[{"name":"College of Air Traffic Control, Civil Aviation Flight University of China, Guanghan, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-1785-008X","authenticated-orcid":false,"given":"Lei","family":"Pan","sequence":"additional","affiliation":[{"name":"School of Computer Science, Civil Aviation Flight University of China, Guanghan, P. R. China"},{"name":"Postdoctoral Workstation of Inspur Group Jinan, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"219","published-online":{"date-parts":[[2025,6,30]]},"reference":[{"doi-asserted-by":"publisher","key":"S021812662550375XBIB001","DOI":"10.1017\/S0047404511000480"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB002","DOI":"10.1093\/deafed\/enz045"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB004","DOI":"10.1109\/CVPR.2018.00812"},{"key":"S021812662550375XBIB005","first-page":"1911","volume-title":"Proc. Ninth Int. Conf. Language Resources and Evaluation","author":"Forster J.","year":"2014"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB006","DOI":"10.1016\/j.cviu.2015.09.013"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB007","DOI":"10.1109\/TMM.2018.2889563"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB008","DOI":"10.1109\/IJCNN60899.2024.10651082"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB011","DOI":"10.4324\/9780429341557-3"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB012","DOI":"10.1109\/TPAMI.2022.3152247"},{"key":"S021812662550375XBIB013","first-page":"232","volume-title":"On Translation","author":"Jakobson R.","year":"1959"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB014","DOI":"10.1163\/9789004495746"},{"volume-title":"Multimodal Discourse: The Modes and Media of Contemporary Communication","year":"2001","author":"Van Leeuwen T.","key":"S021812662550375XBIB015"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB018","DOI":"10.1016\/j.pragma.2005.06.021"},{"key":"S021812662550375XBIB019","first-page":"296","volume-title":"The Meaning of Meaning: A Study of the Influence of Language upon Thought and of the Science of Symbolism","author":"Malinowski B.","year":"1949"},{"key":"S021812662550375XBIB020","first-page":"1597","volume-title":"Proc. 37th Int. Conf. Machine Learning","author":"Chen T.","year":"2020"},{"key":"S021812662550375XBIB021","first-page":"8748","volume-title":"Proc. 38th Int. Conf. Machine Learning","author":"Radford A.","year":"2021"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB022","DOI":"10.3233\/TAD-2008-20207"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB023","DOI":"10.1007\/978-3-030-30493-5_59"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB024","DOI":"10.1109\/TMM.2024.3377095"},{"key":"S021812662550375XBIB025","first-page":"832","volume-title":"Proc. IEEE Conf. Computer Vision and Pattern Recognition","author":"Camg\u00f6z N. C.","year":"2018"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB026","DOI":"10.1049\/iet-cvi.2017.0598"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB027","DOI":"10.1109\/CVPR52729.2023.00249"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB028","DOI":"10.1007\/978-3-642-38628-2_10"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB030","DOI":"10.1016\/j.eswa.2020.113794"},{"key":"S021812662550375XBIB032","first-page":"116","volume":"9","author":"Sahoo A. K.","year":"2014","journal-title":"ARPN J. Eng. Appl. Sci."},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB033","DOI":"10.1038\/s43586-021-00018-1"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB034","DOI":"10.1145\/3427315"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB035","DOI":"10.1109\/ICCV51070.2023.01908"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB037","DOI":"10.1109\/CVPR52733.2024.01738"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB038","DOI":"10.1007\/978-3-031-19815-1_40"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB039","DOI":"10.1109\/CVPR52729.2023.01074"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB040","DOI":"10.1109\/CVPR52688.2022.01129"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB041","DOI":"10.1109\/CVPR52688.2022.01760"},{"volume-title":"Course in General Linguistics","year":"2020","author":"de Saussure F.","key":"S021812662550375XBIB042"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB043","DOI":"10.1109\/CVPR46437.2021.00137"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB044","DOI":"10.1109\/CVPR42600.2020.01004"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB045","DOI":"10.1007\/978-3-030-58586-0_41"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB046","DOI":"10.1109\/TPAMI.2019.2911077"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB047","DOI":"10.1109\/CVPR52688.2022.00506"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB048","DOI":"10.1109\/TMM.2021.3059098"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB049","DOI":"10.1145\/3581783.3611745"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB050","DOI":"10.1109\/CVPR52729.2023.01037"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB051","DOI":"10.1109\/CVPR52688.2022.00507"},{"doi-asserted-by":"publisher","key":"S021812662550375XBIB052","DOI":"10.1109\/TPAMI.2023.3269220"},{"key":"S021812662550375XBIB053","first-page":"17043","volume":"35","author":"Chen Y.","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."}],"container-title":["Journal of Circuits, Systems and Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S021812662550375X","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T06:19:47Z","timestamp":1760681987000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S021812662550375X"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,30]]},"references-count":45,"journal-issue":{"issue":"02","published-print":{"date-parts":[[2026,1,30]]}},"alternative-id":["10.1142\/S021812662550375X"],"URL":"https:\/\/doi.org\/10.1142\/s021812662550375x","relation":{},"ISSN":["0218-1266","1793-6454"],"issn-type":[{"type":"print","value":"0218-1266"},{"type":"electronic","value":"1793-6454"}],"subject":[],"published":{"date-parts":[[2025,6,30]]},"article-number":"2550375"}}