{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T11:13:03Z","timestamp":1775041983423,"version":"3.50.1"},"reference-count":65,"publisher":"World Scientific Pub Co Pte Ltd","issue":"12","funder":[{"DOI":"10.13039\/501100001809","name":"The National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62376127"],"award-info":[{"award-number":["62376127"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"The National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61876089"],"award-info":[{"award-number":["61876089"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"The National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61876185"],"award-info":[{"award-number":["61876185"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"The National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61902281"],"award-info":[{"award-number":["61902281"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"The Natural Science Foundation of Jiangsu Province","award":["BK20141005"],"award-info":[{"award-number":["BK20141005"]}]},{"name":"The Natural Science Foundation of the Jiangsu Higher Education Institutions of China","award":["s14KJB520025"],"award-info":[{"award-number":["s14KJB520025"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Neur. Syst."],"published-print":{"date-parts":[[2024,12]]},"abstract":"<jats:p> In many modern machine learning (ML) models, attention mechanisms (AMs) play a crucial role in processing data and identifying significant parts of the inputs, whether these are text or images. This selective focus enables subsequent stages of the model to achieve improved classification performance. Traditionally, AMs are applied as a preprocessing substructure before a neural network, such as in encoder\/decoder architectures. In this paper, we extend the application of AMs to intermediate stages of data propagation within ML models. Specifically, we propose a generalized attention mechanism (GAM), which can be integrated before each layer of a neural network for classification tasks. The proposed GAM allows for at each layer\/step of the ML architecture identification of the most relevant sections of the intermediate results. Our experimental results demonstrate that incorporating the proposed GAM into various ML models consistently enhances the accuracy of these models. This improvement is achieved with only a marginal increase in the number of parameters, which does not significantly affect the training time. <\/jats:p>","DOI":"10.1142\/s0129065724500631","type":"journal-article","created":{"date-parts":[[2024,8,10]],"date-time":"2024-08-10T15:40:22Z","timestamp":1723304422000},"source":"Crossref","is-referenced-by-count":17,"title":["A Generalized Attention Mechanism to Enhance the Accuracy Performance of Neural Networks"],"prefix":"10.1142","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9625-697X","authenticated-orcid":false,"given":"Pengcheng","family":"Jiang","sequence":"first","affiliation":[{"name":"School of Software, Nanjing University of Information Science and Technology, Nanjing 210044, P.\u00a0R.\u00a0China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6100-6532","authenticated-orcid":false,"given":"Ferrante","family":"Neri","sequence":"additional","affiliation":[{"name":"NICE Research Group, School of Computer Science and Electronic Engineering, University of Surrey, Guildford GU2 7XS, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9069-7547","authenticated-orcid":false,"given":"Yu","family":"Xue","sequence":"additional","affiliation":[{"name":"School of Software, Nanjing University of Information Science and Technology, Nanjing 210044, P.\u00a0R.\u00a0China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1167-0774","authenticated-orcid":false,"given":"Ujjwal","family":"Maulik","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Jadavpur University, Kolkata, India"}]}],"member":"219","published-online":{"date-parts":[[2024,8,31]]},"reference":[{"key":"S0129065724500631BIB001","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2019.2950923"},{"key":"S0129065724500631BIB002","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2021.103293"},{"key":"S0129065724500631BIB003","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2021.3130598"},{"key":"S0129065724500631BIB004","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2023.3243698"},{"key":"S0129065724500631BIB005","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2022.3190448"},{"key":"S0129065724500631BIB006","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065724500059"},{"key":"S0129065724500631BIB007","doi-asserted-by":"publisher","DOI":"10.1111\/mice.13003"},{"key":"S0129065724500631BIB008","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12701"},{"key":"S0129065724500631BIB009","doi-asserted-by":"publisher","DOI":"10.1515\/revneuro-2020-0043"},{"key":"S0129065724500631BIB010","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-023-02032-0"},{"key":"S0129065724500631BIB011","doi-asserted-by":"publisher","DOI":"10.1159\/000512985"},{"key":"S0129065724500631BIB012","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2022.104234"},{"key":"S0129065724500631BIB013","doi-asserted-by":"publisher","DOI":"10.3233\/ICA-230714"},{"key":"S0129065724500631BIB014","doi-asserted-by":"publisher","DOI":"10.1111\/mice.13010"},{"key":"S0129065724500631BIB015","doi-asserted-by":"publisher","DOI":"10.1111\/exsy.12647"},{"key":"S0129065724500631BIB016","doi-asserted-by":"publisher","DOI":"10.3233\/ICA-230711"},{"key":"S0129065724500631BIB017","doi-asserted-by":"publisher","DOI":"10.3233\/ICA-230702"},{"key":"S0129065724500631BIB018","doi-asserted-by":"publisher","DOI":"10.1038\/323533a0"},{"key":"S0129065724500631BIB019","doi-asserted-by":"publisher","DOI":"10.1126\/science.1127647"},{"key":"S0129065724500631BIB020","first-page":"14045","volume":"45","author":"Groenendijk R.","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"S0129065724500631BIB021","first-page":"9593","volume-title":"Advances in Neural Information Processing Systems","volume":"32","author":"Zhang M. R.","year":"2019"},{"key":"S0129065724500631BIB022","first-page":"10334","volume-title":"Proc. 37th Int. Conf. Machine Learning, Proc. Machine Learning Research","volume":"119","author":"Woodworth B.","year":"2020"},{"key":"S0129065724500631BIB023","volume-title":"Int. Conf. Learning Representations","author":"Kingma D. P.","year":"2015"},{"key":"S0129065724500631BIB024","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2023.106504"},{"key":"S0129065724500631BIB025","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065723500077"},{"key":"S0129065724500631BIB026","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2023.111084"},{"key":"S0129065724500631BIB027","doi-asserted-by":"publisher","DOI":"10.1142\/S012906572450014X"},{"key":"S0129065724500631BIB028","doi-asserted-by":"publisher","DOI":"10.1142\/S0218213022400085"},{"key":"S0129065724500631BIB029","doi-asserted-by":"publisher","DOI":"10.1007\/s41095-022-0271-y"},{"key":"S0129065724500631BIB030","doi-asserted-by":"publisher","DOI":"10.1111\/exsy.12494"},{"key":"S0129065724500631BIB031","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-019-04146-4"},{"key":"S0129065724500631BIB032","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-019-04359-7"},{"key":"S0129065724500631BIB033","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2017.2682102"},{"key":"S0129065724500631BIB034","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12852"},{"key":"S0129065724500631BIB035","volume-title":"Int. Conf. Learning Representations","author":"Bahdanau D.","year":"2015"},{"key":"S0129065724500631BIB036","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1166"},{"key":"S0129065724500631BIB037","first-page":"17723","volume":"34","author":"Zhu C.","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"S0129065724500631BIB038","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1423"},{"key":"S0129065724500631BIB039","first-page":"1877","volume":"33","author":"Brown T.","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"S0129065724500631BIB040","doi-asserted-by":"publisher","DOI":"10.1111\/mice.12695"},{"key":"S0129065724500631BIB041","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2019.2913372"},{"key":"S0129065724500631BIB042","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"S0129065724500631BIB043","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2022.3211171"},{"key":"S0129065724500631BIB044","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW56347.2022.00309"},{"key":"S0129065724500631BIB045","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00122"},{"key":"S0129065724500631BIB046","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00995"},{"key":"S0129065724500631BIB047","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2023.3268446"},{"key":"S0129065724500631BIB048","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065723500107"},{"key":"S0129065724500631BIB049","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065723500193"},{"key":"S0129065724500631BIB050","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065723500612"},{"key":"S0129065724500631BIB051","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065723500545"},{"key":"S0129065724500631BIB052","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065723500314"},{"key":"S0129065724500631BIB053","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065723500260"},{"key":"S0129065724500631BIB054","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065723500685"},{"key":"S0129065724500631BIB055","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2019.2943010"},{"key":"S0129065724500631BIB056","author":"Simonyan K.","year":"2015","journal-title":"Int. Conf. Learning Representations"},{"key":"S0129065724500631BIB057","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"S0129065724500631BIB058","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01264-9_8"},{"key":"S0129065724500631BIB059","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00255"},{"key":"S0129065724500631BIB060","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01044"},{"key":"S0129065724500631BIB061","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.01587"},{"key":"S0129065724500631BIB062","author":"Trockman A.","year":"2023","journal-title":"Trans. Mach. Learn. Res."},{"key":"S0129065724500631BIB063","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.00532"},{"key":"S0129065724500631BIB064","doi-asserted-by":"publisher","DOI":"10.1145\/3065386"},{"key":"S0129065724500631BIB065","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00716"}],"container-title":["International Journal of Neural Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0129065724500631","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,22]],"date-time":"2024-10-22T03:53:35Z","timestamp":1729569215000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S0129065724500631"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,31]]},"references-count":65,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2024,12]]}},"alternative-id":["10.1142\/S0129065724500631"],"URL":"https:\/\/doi.org\/10.1142\/s0129065724500631","relation":{},"ISSN":["0129-0657","1793-6462"],"issn-type":[{"value":"0129-0657","type":"print"},{"value":"1793-6462","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,31]]},"article-number":"2450063"}}