{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T13:36:48Z","timestamp":1753882608768,"version":"3.41.2"},"reference-count":32,"publisher":"World Scientific Pub Co Pte Ltd","issue":"12","funder":[{"name":"the Scientific Research Fund of Liaoning Provincial Edu cation Department","award":["LJ212414435009"],"award-info":[{"award-number":["LJ212414435009"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J CIRCUIT SYST COMP"],"published-print":{"date-parts":[[2025,8]]},"abstract":"<jats:p> The Transformer model has achieved notable success in various sequence generation tasks such as machine translation and generative text summarization. However, its performance remains suboptimal in low-resource scenarios. This limitation stems from insufficient learning of model parameters and the reliance on a single-scale, global modeling mechanism. Inspired by multi-scale approaches in computer vision and the dot-product mechanism in Transformer attention, we propose a simple yet parameter-efficient method to enhance the model\u2019s learning capacity without increasing its parameter count. The method involves constructing multi-scale features with diverse attributes from the model\u2019s hidden dimension through matrix decomposition-based sampling, followed by the fusion of attention weights generated from these multi-scale features. This fusion results in a more robust data representation, allowing the model to learn from the data more effectively. Experimental results show that our method outperforms baseline approaches by an average of 0.45 BLEU points across six machine translation tasks and achieves a 0.55 ROUGE point improvement on the CNN-DailyMail summarization test set. <\/jats:p>","DOI":"10.1142\/s0218126625502913","type":"journal-article","created":{"date-parts":[[2025,3,11]],"date-time":"2025-03-11T09:55:36Z","timestamp":1741686936000},"source":"Crossref","is-referenced-by-count":0,"title":["Multi-Scale Feature Fusion Using Matrix Decomposition for Enhanced Transformer Performance on Small-Scale Datasets"],"prefix":"10.1142","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4468-9706","authenticated-orcid":false,"given":"Murun","family":"Yang","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, Northeastern University, Heping District, No. 11, Lane 3, Wenhua Road, Heping District, Shenyang, Liaoning, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2878-6119","authenticated-orcid":false,"given":"Fuxue","family":"Li","sequence":"additional","affiliation":[{"name":"College of Electrical Engineering, Yingkou Institute of Technology, BoWen Road, Yingkou, Liaoning, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5842-6501","authenticated-orcid":false,"given":"Tong","family":"Xiao","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Northeastern University, Heping District, No. 11, Lane 3, Wenhua Road, Heping District, Shenyang, Liaoning, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6537-7007","authenticated-orcid":false,"given":"Jingbo","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Northeastern University, Heping District, No. 11, Lane 3, Wenhua Road, Heping District, Shenyang, Liaoning, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"219","published-online":{"date-parts":[[2025,5,12]]},"reference":[{"key":"S0218126625502913BIB001","first-page":"3104","volume-title":"Advances in Neural Information Processing Systems 27: Annual Conf. Neural Information Processing Systems 2014","author":"Sutskever I.","year":"2014"},{"volume-title":"3rd Int. Conf. Learning Representations, ICLR 2015, Conf. Track Proc.","year":"2015","author":"Bahdanau D.","key":"S0218126625502913BIB002"},{"key":"S0218126625502913BIB003","first-page":"1243","volume-title":"Proc. 34th Int. Conf. Machine Learning, ICML 2017","author":"Gehring J.","year":"2017"},{"key":"S0218126625502913BIB004","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems 30: Annual Conf. Neural Information Processing Systems 2017","author":"Vaswani A.","year":"2017"},{"key":"S0218126625502913BIB005","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-3204"},{"key":"S0218126625502913BIB007","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.10913"},{"key":"S0218126625502913BIB008","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1176"},{"key":"S0218126625502913BIB009","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.40"},{"key":"S0218126625502913BIB010","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.72"},{"key":"S0218126625502913BIB013","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1475"},{"key":"S0218126625502913BIB014","doi-asserted-by":"publisher","DOI":"10.3390\/app9163389"},{"key":"S0218126625502913BIB015","doi-asserted-by":"publisher","DOI":"10.3390\/app12136518"},{"key":"S0218126625502913BIB017","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1082"},{"key":"S0218126625502913BIB020","doi-asserted-by":"publisher","DOI":"10.1109\/TCE.2023.3332878"},{"key":"S0218126625502913BIB023","first-page":"311","volume-title":"Proc. 40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni K.","year":"2002"},{"key":"S0218126625502913BIB024","first-page":"74","volume-title":"Text Summarization Branches Out","author":"Lin C.-Y.","year":"2004"},{"key":"S0218126625502913BIB026","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-4009"},{"volume-title":"3rd Int. Conf. Learning Representations, ICLR 2015, Conf. Track Proc.","year":"2015","author":"Kingma D. P.","key":"S0218126625502913BIB027"},{"volume-title":"9th Int. Conf. Learning Representations, ICLR","year":"2021","author":"Mehta S.","key":"S0218126625502913BIB028"},{"volume-title":"8th Int. Conf. Learning Representations, ICLR 2020","year":"2020","author":"Zhu J.","key":"S0218126625502913BIB029"},{"key":"S0218126625502913BIB030","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.135"},{"volume-title":"9th Int. Conf.Learning Representations, ICLR 2021","year":"2021","author":"Liu X.","key":"S0218126625502913BIB032"},{"key":"S0218126625502913BIB034","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01252-6_24"},{"key":"S0218126625502913BIB035","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00675"},{"key":"S0218126625502913BIB036","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2023.3268659"},{"key":"S0218126625502913BIB037","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2023.3337897"},{"key":"S0218126625502913BIB038","doi-asserted-by":"publisher","DOI":"10.1109\/TSC.2024.3404347"},{"key":"S0218126625502913BIB039","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"S0218126625502913BIB040","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00476"},{"key":"S0218126625502913BIB041","first-page":"1","author":"Yu K.","year":"2024","journal-title":"IEEE J. Biomed. Health Inf."},{"key":"S0218126625502913BIB042","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2024.3414496"},{"key":"S0218126625502913BIB044","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1003"}],"container-title":["Journal of Circuits, Systems and Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0218126625502913","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T06:34:36Z","timestamp":1750314876000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S0218126625502913"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,12]]},"references-count":32,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2025,8]]}},"alternative-id":["10.1142\/S0218126625502913"],"URL":"https:\/\/doi.org\/10.1142\/s0218126625502913","relation":{},"ISSN":["0218-1266","1793-6454"],"issn-type":[{"type":"print","value":"0218-1266"},{"type":"electronic","value":"1793-6454"}],"subject":[],"published":{"date-parts":[[2025,5,12]]},"article-number":"2550291"}}