{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T19:16:28Z","timestamp":1790190988214,"version":"4.1.0"},"reference-count":51,"publisher":"World Scientific Pub Co Pte Ltd","issue":"07","funder":[{"DOI":"10.13039\/501100004543","name":"China Scholarship Council","doi-asserted-by":"crossref","award":["202308060192"],"award-info":[{"award-number":["202308060192"]}],"id":[{"id":"10.13039\/501100004543","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Neur. Syst."],"published-print":{"date-parts":[[2026,7]]},"abstract":"<jats:p>Fine-tuning large-scale Transformer-based models is computationally expensive due to the enormous parameter space. Low-Rank Adaptation (LoRA) substantially reduces the number of trainable parameters while maintaining performance; however, identifying the optimal LoRA configuration \u2014 such as rank r, scaling factor [Formula: see text], and insertion positions \u2014 remains challenging. To address this issue, we propose a zero-shot proxy metric, termed Gradient Projection Score (GPS), which enables rapid evaluation of candidate configurations using only a few forward and backward passes. Building upon this metric, we further introduce EvoLoRA, a zero-shot evolutionary architecture search method that jointly optimizes three objectives: performance proxy, evaluation stability, and trainable parameter size. EvoLoRA automatically discovers effective LoRA configurations across different models and datasets. Experimental results demonstrate that GPS is strongly correlated with final model performance; moreover, on tasks such as image classification and object detection, EvoLoRA markedly reduces search and training costs while generally outperforming other fine-tuning methods and manually designed LoRA configurations.<\/jats:p>","DOI":"10.1142\/s0129065726500255","type":"journal-article","created":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T09:51:01Z","timestamp":1770976261000},"source":"Crossref","is-referenced-by-count":1,"title":["Zero-Shot Evolutionary Architecture Search for Low-Rank Adaptation"],"prefix":"10.1142","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-8988-7661","authenticated-orcid":false,"given":"Pengjin","family":"Wu","sequence":"first","affiliation":[{"name":"School of Computer Science and Electronic Engineering, University of Surrey, Guildford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6100-6532","authenticated-orcid":false,"given":"Ferrante","family":"Neri","sequence":"additional","affiliation":[{"name":"School of Computer Science and Electronic Engineering, University of Surrey, Guildford, UK"},{"name":"School of Software, Nanjing University of Information Science and Technology, Nanjing, P. R. China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4485-4249","authenticated-orcid":false,"given":"Zhenhua","family":"Feng","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, P. R. China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"219","published-online":{"date-parts":[[2026,3,19]]},"reference":[{"key":"S0129065726500255BIB001","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065709002002"},{"key":"S0129065726500255BIB002","first-page":"11","volume":"30","author":"Vaswani A.","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"S0129065726500255BIB003","doi-asserted-by":"publisher","DOI":"10.1016\/0096-3003(94)90134-1"},{"issue":"2","key":"S0129065726500255BIB004","first-page":"155","volume":"7","author":"Adeli H.","year":"1993","journal-title":"Int. J. Supercomput. Appl."},{"key":"S0129065726500255BIB005","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-023-00626-4"},{"key":"S0129065726500255BIB006","volume-title":"Int. Conf. on Learning Representations","author":"Hu E. J.","year":"2022"},{"key":"S0129065726500255BIB007","first-page":"1877","volume":"33","author":"Brown T.","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"S0129065726500255BIB008","unstructured":"J. Devlin, M.W. Chang, K. Lee and K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, in\n                      Proc. of the 2019 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)\n                      (Association for Computational Linguistics, 2019), pp. 4171\u20134186."},{"key":"S0129065726500255BIB009","volume-title":"Int. Conf. on Learning Representations","author":"Dosovitskiy A.","year":"2020"},{"key":"S0129065726500255BIB010","first-page":"8748","volume-title":"Int. Conf. on Machine Learning","author":"Radford A.","year":"2021"},{"key":"S0129065726500255BIB011","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58452-8_13"},{"key":"S0129065726500255BIB012","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065723500351"},{"key":"S0129065726500255BIB013","volume-title":"Int. Conf. on Learning Representations","author":"Zhu X.","year":"2021"},{"key":"S0129065726500255BIB014","doi-asserted-by":"crossref","unstructured":"E. B. Zaken, Y. Goldberg and S. Ravfogel, Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models, in\n                      Proc. 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)\n                      (Association for Computational Linguistics, 2022), pp. 1\u20139.","DOI":"10.18653\/v1\/2022.acl-short.1"},{"key":"S0129065726500255BIB015","volume-title":"The Twelfth Int. Conf. on Learning Representations","author":"Zhao B.","year":"2024"},{"key":"S0129065726500255BIB016","first-page":"2790","volume-title":"Int. Conf. on Machine Learning","author":"Houlsby N.","year":"2019"},{"key":"S0129065726500255BIB017","doi-asserted-by":"crossref","unstructured":"X. L. Li and P. Liang, Prefix-tuning: Optimizing continuous prompts for generation, in\n                      Proc. 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int. Joint Conf. on Natural Language Processing (Volume 1: Long Papers)\n                      (Association for Computational Linguistics, 2021), pp. 4582\u20134597.","DOI":"10.18653\/v1\/2021.acl-long.353"},{"key":"S0129065726500255BIB018","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.243"},{"key":"S0129065726500255BIB019","first-page":"17783","volume-title":"Proc. 41st Int. Conf. on Machine Learning","author":"Hayou S.","year":"2024"},{"key":"S0129065726500255BIB020","volume-title":"Int. Conf. on Learning Representations","author":"Zhang Q.","year":"2023"},{"key":"S0129065726500255BIB021","first-page":"32100","volume-title":"Int. Conf. on Machine Learning","author":"Liu S.-Y.","year":"2024"},{"key":"S0129065726500255BIB022","doi-asserted-by":"publisher","DOI":"10.52202\/075280-0441"},{"key":"S0129065726500255BIB023","first-page":"1022","volume":"34","author":"Karimi Mahabadi R.","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"S0129065726500255BIB024","volume-title":"Int. Conf. on Learning Representations","author":"Zoph B.","year":"2017"},{"issue":"01","key":"S0129065726500255BIB025","first-page":"4780","volume":"33","author":"Real E.","year":"2019","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"S0129065726500255BIB026","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065721500350"},{"key":"S0129065726500255BIB027","volume-title":"Int. Conf. on Learning Representations","author":"Liu H.","year":"2019"},{"key":"S0129065726500255BIB028","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN64981.2025.11229286"},{"key":"S0129065726500255BIB029","volume-title":"Int. Conf. on Learning Representations","author":"Abdelfattah M. S.","year":"2021"},{"key":"S0129065726500255BIB030","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065723500168"},{"key":"S0129065726500255BIB031","volume-title":"Int. Conf. on Learning Representations","author":"Lee N.","year":"2019"},{"key":"S0129065726500255BIB032","volume-title":"Int. Conf. on Learning Representations","author":"Wang C.","year":"2020"},{"key":"S0129065726500255BIB033","first-page":"6377","volume":"33","author":"Tanaka H.","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"S0129065726500255BIB034","first-page":"7588","volume-title":"Int. Conf. on Machine Learning","author":"Mellor J.","year":"2021"},{"key":"S0129065726500255BIB035","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-86383-8_44"},{"key":"S0129065726500255BIB036","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00040"},{"key":"S0129065726500255BIB037","volume-title":"Int. Conf. on Learning Representations","author":"Li G.","year":"2023"},{"key":"S0129065726500255BIB038","doi-asserted-by":"publisher","DOI":"10.1109\/72.329686"},{"key":"S0129065726500255BIB039","doi-asserted-by":"publisher","DOI":"10.1080\/03052150108940930"},{"key":"S0129065726500255BIB040","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcsr.2013.07.027"},{"key":"S0129065726500255BIB041","doi-asserted-by":"publisher","DOI":"10.1002\/9781118534823"},{"key":"S0129065726500255BIB042","doi-asserted-by":"publisher","DOI":"10.1142\/S012906572450014X"},{"key":"S0129065726500255BIB043","doi-asserted-by":"publisher","DOI":"10.1109\/4235.996017"},{"key":"S0129065726500255BIB044","unstructured":"A. Krizhevsky, Learning multiple layers of features from tiny images, Technical report, University of Toronto, Toronto, Ontario (2009), https:\/\/www.cs.toronto.edu\/~kriz\/learning-features-2009-TR.pdf."},{"key":"S0129065726500255BIB045","doi-asserted-by":"publisher","DOI":"10.1109\/ICVGIP.2008.47"},{"key":"S0129065726500255BIB046","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2012.6248092"},{"key":"S0129065726500255BIB047","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5539970"},{"key":"S0129065726500255BIB048","first-page":"7","volume-title":"NIPS Workshop on Deep Learning and Unsupervised Feature Learning","author":"Netzer Y.","year":"2011"},{"key":"S0129065726500255BIB049","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"S0129065726500255BIB050","first-page":"1597","volume-title":"Int. Conf. on Machine Learning","author":"Chen T.","year":"2020"},{"key":"S0129065726500255BIB051","doi-asserted-by":"crossref","unstructured":"J. Howard and S. Ruder, Universal language model fine-tuning for text classification, in\n                      Proc. 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)\n                      (Association for Computational Linguistics, 2018), pp. 328\u2013339.","DOI":"10.18653\/v1\/P18-1031"}],"container-title":["International Journal of Neural Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0129065726500255","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T05:39:40Z","timestamp":1775626780000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S0129065726500255"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,19]]},"references-count":51,"journal-issue":{"issue":"07","published-print":{"date-parts":[[2026,7]]}},"alternative-id":["10.1142\/S0129065726500255"],"URL":"https:\/\/doi.org\/10.1142\/s0129065726500255","relation":{},"ISSN":["0129-0657","1793-6462"],"issn-type":[{"value":"0129-0657","type":"print"},{"value":"1793-6462","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,19]]},"article-number":"2650025"}}