{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,4]],"date-time":"2026-11-04T19:55:23Z","timestamp":1793822123959,"version":"4.6.1"},"reference-count":18,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T00:00:00Z","timestamp":1764633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Big Data"],"abstract":"<jats:p>\n                    The successful application of large-scale transformer models in Natural Language Processing (NLP) is often hindered by the substantial computational cost and data requirements of full fine-tuning. This challenge is particularly acute in low-resource settings, where standard fine-tuning can lead to catastrophic overfitting and model collapse. To address this, Parameter-Efficient Fine-Tuning (PEFT) methods have emerged as a promising solution. However, a direct comparative analysis of their trade-offs under unified low-resource conditions is lacking. This study provides a rigorous empirical evaluation of three prominent PEFT methods: Low-Rank Adaptation (LoRA), Infused Adapter by Inhibiting and Amplifying Inner Activations (IA\n                    <jats:sup>3<\/jats:sup>\n                    ), and a Representation Fine-Tuning (ReFT) strategy. Using a DistilBERT base model on low-resource versions of the AG News and Amazon Reviews datasets, the present work compares these methods against a full fine-tuning baseline across accuracy, F1 score, trainable parameters, and GPU memory usage. The findings reveal that while all PEFT methods dramatically outperform the baseline, LoRA consistently achieves the highest F1 scores (0.909 on Amazon Reviews). Critically, ReFT delivers nearly identical performance (~98% of LoRA's F1 score) while training only ~3% of the parameters, establishing it as the most efficient method. This research demonstrates that PEFT is not merely an efficiency optimization, but a necessary tool for robust generalization in data-scarce environments, providing practitioners with a clear guide to navigate the performance\u2014efficiency trade-off. By unifying these evaluations under controlled conditions, this study advances beyond fragmented prior research and offers a systematic framework for selecting PEFT strategies.\n                  <\/jats:p>","DOI":"10.3389\/fdata.2025.1677331","type":"journal-article","created":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T05:14:37Z","timestamp":1764652477000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Parameter-efficient fine-tuning for low-resource text classification: a comparative study of LoRA, IA3, and ReFT"],"prefix":"10.3389","volume":"8","author":[{"given":"Steve","family":"Nwaiwu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,12,2]]},"reference":[{"key":"B1","article-title":"Scaling sparse fine-tuning to large language models","author":"Ansell","year":"2024","journal-title":"arXiv preprint, arXiv:2401.16405"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18653\/v1\/2022.acl-short.1","article-title":"\u201cBitfit: simple parameter-efficient fine-tuning for transformer-based masked language-models,\u201d","author":"Ben Zaken","year":"2022","journal-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)"},{"key":"B3","article-title":"Lotr: Low tensor rank weight adaptation","author":"Bershatsky","year":"2024","journal-title":"arXiv preprint, arXiv:2402.01376"},{"key":"B4","first-page":"4171","article-title":"\u201cBert: pre-training of deep bidirectional transformers for language understanding,\u201d","author":"Devlin","year":"2019","journal-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)"},{"key":"B5","article-title":"Context-peft: efficient multi-modal, multi-task fine-tuning","author":"Hadji-Kyriacou","year":"2023","journal-title":"arXiv preprint, arXiv:2312.08900"},{"key":"B6","first-page":"2790","article-title":"\u201cParameter-efficient transfer learning for NLP,\u201d","author":"Houlsby","year":"2019","journal-title":"Proceedings of the 36th International Conference on Machine Learning"},{"key":"B7","article-title":"Lora: Low-rank adaptation of large language models","author":"Hu","year":"2021","journal-title":"arXiv preprint, arXiv:2106.09685"},{"key":"B8","author":"Li","year":"2024","journal-title":"Ag News and IMDB"},{"key":"B9","article-title":"Scaling down to scale up: a guide to parameter-efficient fine-tuning","author":"Lialin","year":"2024","journal-title":"arXiv preprint, arXiv:2303.15647v152"},{"key":"B10","first-page":"1950","article-title":"\u201cFew-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,\u201d","author":"Liu","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B11","article-title":"Hybrid and unitary fine-tuning of large language models: Methods and benchmarking under resource constraints","author":"Qi","year":"2025","journal-title":"arXiv preprint, arXiv:2507.18076"},{"key":"B12","article-title":"Distilbert: a distilled version of bert: smaller, faster, cheaper and lighter","author":"Sanh","year":"2019","journal-title":"arXiv preprint, arXiv:1910.01108"},{"key":"B13","first-page":"78244","article-title":"\u201cStructured unrestricted-rank matrices for parameter efficient finetuning,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Sehanobish","year":"2024"},{"key":"B14","first-page":"5998","article-title":"\u201cAttention is all you need,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani","year":"2017"},{"key":"B15","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1007\/s10462-025-11236-4","article-title":"Parameter-efficient fine-tuning in large language models: a survey of methodologies","volume":"58","author":"Wang","year":"2025","journal-title":"Artif. Intell. Rev"},{"key":"B16","first-page":"63908","article-title":"\u201cReft: Representation finetuning for language models,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Wu","year":"2024"},{"key":"B17","article-title":"Parameter-efficient fine-tuning for foundation models","author":"Zhang","year":"2025","journal-title":"arXiv preprint, arXiv:2501.13787"},{"key":"B18","first-page":"649","article-title":"\u201cCharacter-level convolutional networks for text classification,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Zhang","year":"2015"}],"container-title":["Frontiers in Big Data"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2025.1677331\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T05:14:39Z","timestamp":1764652479000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2025.1677331\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,2]]},"references-count":18,"alternative-id":["10.3389\/fdata.2025.1677331"],"URL":"https:\/\/doi.org\/10.3389\/fdata.2025.1677331","relation":{},"ISSN":["2624-909X"],"issn-type":[{"value":"2624-909X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,2]]},"article-number":"1677331"}}