{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T13:36:15Z","timestamp":1785418575710,"version":"3.56.0"},"reference-count":23,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T00:00:00Z","timestamp":1778284800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62262005"],"award-info":[{"award-number":["62262005"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"award":["62262005"],"award-info":[{"award-number":["62262005"]}],"id":[{"id":"https:\/\/ror.org\/01h0zpd94","id-type":"ROR","asserted-by":"publisher"}]},{"name":"High-level Innovative Talents in Guizhou Province","award":["GCC[2023]033"],"award-info":[{"award-number":["GCC[2023]033"]}]},{"DOI":"10.13039\/501100010828","name":"Department of Education of Guizhou Province","doi-asserted-by":"crossref","award":["QJJ[2024]009"],"award-info":[{"award-number":["QJJ[2024]009"]}],"id":[{"id":"10.13039\/501100010828","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100010828","name":"Department of Education of Guizhou Province","doi-asserted-by":"crossref","award":["QJJ[2023]011"],"award-info":[{"award-number":["QJJ[2023]011"]}],"id":[{"id":"10.13039\/501100010828","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Ministry of Education","award":["EBME25-F-12"],"award-info":[{"award-number":["EBME25-F-12"]}]},{"award":["EBME25-F-12"],"award-info":[{"award-number":["EBME25-F-12"]}],"id":[{"id":"https:\/\/ror.org\/00b3tsf98","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Parameter-efficient fine-tuning (PEFT) enables large language models to adapt to downstream tasks with low computational cost. As a representative high-rank PEFT method, MoRA (High-Rank Updating for Parameter-Efficient Fine-Tuning) improves update expressiveness through a compression\u2013transformation\u2013decompression reparameterization mechanism. However, its bottleneck subspace is still modeled using a freely learned linear transformation. In addition, grouped compression may project information from different original directions into shared bottleneck coordinates. This may reduce subspace separability and lead to inefficient utilization of the effective update space. To address this limitation, we propose GL-log-MoRA, which introduces a learnable general linear transformation into the MoRA bottleneck subspace and applies log-determinant regularization to encourage a more balanced spectral structure. In this way, the proposed method improves directional coordination and subspace expressiveness without imposing hard structural constraints or causing noticeable memory overhead. We evaluate GL-log-MoRA on five benchmarks: LogiQA, Financial PhraseBank, GSM8K, FinQA, and HotpotQA. The results show that GL-log-MoRA achieves the best performance on these downstream tasks and yields small but consistent improvements over MoRA under the same parameter budget. Compared with MoRA, GL-log-MoRA improves LogiQA from 42.50% to 45.45% and Financial PhraseBank from 81.60% to 83.02%. It also improves GSM8K from 63.1% to 64.6%, FinQA from 10.02% to 10.23%, and HotpotQA from 70.6% to 70.8%. Meanwhile, the average empirical effective-rank indicator increases from 1.05 to 2.80. Peak GPU memory changes only slightly, from 18.21 GB to 18.28 GB.<\/jats:p>","DOI":"10.3390\/info17050460","type":"journal-article","created":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T07:49:09Z","timestamp":1778572149000},"page":"460","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Parameter-Efficient Fine-Tuning via General Linear Structural Regularization for High-Rank Adaptation"],"prefix":"10.3390","volume":"17","author":[{"given":"Bo","family":"Zhao","sequence":"first","affiliation":[{"name":"School of Big Data and Computer Science, Guizhou Normal University, Guiyang 550025, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weihua","family":"Ou","sequence":"additional","affiliation":[{"name":"School of Big Data and Computer Science, Guizhou Normal University, Guiyang 550025, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,5,9]]},"reference":[{"key":"ref_1","first-page":"1877","article-title":"Language Models Are Few-Shot Learners","volume":"33","author":"Brown","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_2","unstructured":"Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., and Vaughan, A. (2024). The Llama 3 Herd of Models. arXiv."},{"key":"ref_3","unstructured":"Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2022). LoRA: Low-Rank Adaptation of Large Language Models. arXiv."},{"key":"ref_4","first-page":"2790","article-title":"Parameter-Efficient Transfer Learning for NLP","volume":"Volume 97","author":"Houlsby","year":"2019","journal-title":"Proceedings of the 36th International Conference on Machine Learning (ICML)"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Lester, B., Al-Rfou, R., and Constant, N. (2021). The Power of Scale for Parameter-Efficient Prompt Tuning. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics.","DOI":"10.18653\/v1\/2021.emnlp-main.243"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, X.L., and Liang, P. (2021). Prefix-Tuning: Optimizing Continuous Prompts for Generation. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Association for Computational Linguistics.","DOI":"10.18653\/v1\/2021.acl-long.353"},{"key":"ref_7","first-page":"1950","article-title":"Few-Shot Parameter-Efficient Fine-Tuning Is Better and Cheaper than In-Context Learning","volume":"Volume 35","author":"Liu","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_8","unstructured":"Zhang, Q., Chen, M., Bukharin, A., Karampatziakis, N., He, P., Cheng, Y., Chen, W., and Zhao, T. (2023, January 1\u20135). AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning. Proceedings of the 11th International Conference on Learning Representations (ICLR), Kigali, Rwanda."},{"key":"ref_9","first-page":"32100","article-title":"DoRA: Weight-Decomposed Low-Rank Adaptation","volume":"Volume 235","author":"Liu","year":"2024","journal-title":"Proceedings of the 41st International Conference on Machine Learning (ICML)"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"10088","DOI":"10.52202\/075280-0441","article-title":"QLoRA: Efficient Finetuning of Quantized LLMs","volume":"Volume 36","author":"Dettmers","year":"2023","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_11","unstructured":"Lialin, V., Deshpande, V., and Rumshisky, A. (2024, January 7\u201311). ReLoRA: High-Rank Training through Low-Rank Updates. Proceedings of the 12th International Conference on Learning Representations (ICLR), Vienna, Austria."},{"key":"ref_12","unstructured":"Jiang, T., Huang, S., Luo, S., Zhang, Z., Huang, H., Wei, F., Deng, W., Sun, F., Zhang, Q., and Wang, D. (2024). MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning. arXiv."},{"key":"ref_13","unstructured":"Albert, P., Zhang, F.Z., Saratchandran, H., Rodriguez-Opazo, C., van den Hengel, A., and Abbasnejad, E. (2025, January 24\u201328). RandLoRA: Full-Rank Parameter-Efficient Fine-Tuning of Large Models. Proceedings of the 13th International Conference on Learning Representations (ICLR), Singapore."},{"key":"ref_14","unstructured":"Huang, Q., Ko, T., Zhuang, Z., Tang, L., and Zhang, Y. (2025, January 24). HiRA: Parameter-Efficient Hadamard High-Rank Adaptation for Large Language Models. Proceedings of the Thirteenth International Conference on Learning Representations (ICLR), Singapore."},{"key":"ref_15","unstructured":"Singhal, R., Ponkshe, K., Vartak, R., and Vepakomma, P. (2026, January 23\u201327). ABBA-Adapters: Efficient and Expressive Fine-Tuning of Foundation Models. Proceedings of the 14th International Conference on Learning Representations (ICLR), Rio de Janeiro, Brazil."},{"key":"ref_16","unstructured":"Gurung, A., and Campbell, J. (2026). HyperAdapt: Simple High-Rank Adaptation. Transactions on Machine Learning Research. arXiv."},{"key":"ref_17","first-page":"41425","article-title":"SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors","volume":"37","author":"Lingam","year":"2024","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wang, X., Chen, T., Ge, Q., Xia, H., Bao, R., Zheng, R., Zhang, Q., Gui, T., and Huang, X. (2023). Orthogonal Subspace Learning for Language Model Continual Learning. Findings of the Association for Computational Linguistics: EMNLP 2023, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2023.findings-emnlp.715"},{"key":"ref_19","unstructured":"Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., and Nakano, R. (2021). Training Verifiers to Solve Math Word Problems. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Liu, J., Cui, L., Liu, H., Huang, D., Wang, Y., and Zhang, Y. (2020). LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning. Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20), International Joint Conferences on Artificial Intelligence Organization.","DOI":"10.24963\/ijcai.2020\/501"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Chen, Z., Chen, W., Smiley, C., Shah, S., Borova, I., Langdon, D., Moussa, R., Beane, M., Huang, T.-H., and Routledge, B. (2021). FinQA: A Dataset of Numerical Reasoning over Financial Data. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics.","DOI":"10.18653\/v1\/2021.emnlp-main.300"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W.W., Salakhutdinov, R., and Manning, C.D. (2018). HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics.","DOI":"10.18653\/v1\/D18-1259"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"782","DOI":"10.1002\/asi.23062","article-title":"Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts","volume":"65","author":"Malo","year":"2014","journal-title":"J. Assoc. Inf. Sci. Technol."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/17\/5\/460\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,13]],"date-time":"2026-05-13T04:13:53Z","timestamp":1778645633000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/17\/5\/460"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,9]]},"references-count":23,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2026,5]]}},"alternative-id":["info17050460"],"URL":"https:\/\/doi.org\/10.3390\/info17050460","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,9]]}}}