{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T16:50:32Z","timestamp":1781715032908,"version":"3.54.5"},"reference-count":28,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T00:00:00Z","timestamp":1772496000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Fine-tuning a BERT-Base model for specific tasks, such as sentiment analysis, has become resource-intensive and often requires high computational power and memory. This paper introduces SCALE, a novel resource-efficient fine-tuning method that targets the most critical transformer layers, which reduces computational costs without sacrificing performance. By dynamically profiling transformer layers via activation magnitudes and attention entropy, SCALE selects and adapts only the most influential layers with lightweight adapter modules. The proposed method outperforms traditional fine-tuning techniques, achieving a 2.3% improvement in accuracy on the IMDB dataset and reducing training time by 56.3% compared to full-model fine-tuning. Experiments across various sentiment analysis benchmarks demonstrate SCALE\u2019s effectiveness in optimizing fine-tuning for the BERT-base model in resource-constrained environments, achieving up to 99% of the performance of full-model fine-tuning while using only 40% of the parameters. The empirical validation in this study is restricted to binary and multi-class sentiment classification. The evaluation specifically reflects effectiveness in sentiment analysis text classification tasks.<\/jats:p>","DOI":"10.3390\/computers15030159","type":"journal-article","created":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T09:45:12Z","timestamp":1772531112000},"page":"159","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["A Resource-Efficient Approach to Fine-Tuning a BERT-Base Model for Sentiment Analysis"],"prefix":"10.3390","volume":"15","author":[{"given":"Abdullah M.","family":"Basahel","sequence":"first","affiliation":[{"name":"Faculty of Economics and Administration, King Abdulaziz University, Jeddah 21589, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shreyanth H.","family":"Giriyappa","sequence":"additional","affiliation":[{"name":"School of Computer Science and Mathematics, Liverpool John Moores University, Liverpool L3 3AF, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8621-4880","authenticated-orcid":false,"given":"Furqan","family":"Alam","sequence":"additional","affiliation":[{"name":"Faculty of Computing and Information Technology (FoCIT), Sohar University, Sohar 311, Oman"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tahani Saleh Mohammed","family":"Alnazzawi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Engineering, Taibah University, Madinah 41477, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Saqib","family":"Qamar","sequence":"additional","affiliation":[{"name":"Faculty of Computing and Information Technology (FoCIT), Sohar University, Sohar 311, Oman"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adnan Ahmed","family":"Abi Sen","sequence":"additional","affiliation":[{"name":"Hussein ElSayyed Research Center, Deanship of Graduate Studies & Scientific Research, University of Prince Mugrin, Madinah 42241, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,3,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"100048","DOI":"10.1016\/j.nlp.2023.100048","article-title":"A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4","volume":"6","author":"Kalyan","year":"2024","journal-title":"Nat. Lang. Process. J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1007\/s10462-024-10888-y","article-title":"Large Language Models (LLMs): Survey, Technical Frameworks, and Future Challenges","volume":"57","author":"Kumar","year":"2024","journal-title":"Artif. Intell. Rev."},{"key":"ref_3","unstructured":"Parthasarathy, V.B., Zafar, A., Khan, A., and Shahid, A. (2024). The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities. arXiv."},{"key":"ref_4","unstructured":"Zhang, S., Dong, L., Li, X., Zhang, S., Sun, X., Wang, S., Li, J., Hu, R., Zhang, T., and Wu, F. (2023). Instruction Tuning for Large Language Models: A Survey. arXiv."},{"key":"ref_5","unstructured":"Golnari, P.A., and Wang, S. (2023). LoRA-Enhanced Distillation on Guided Diffusion Models. arXiv."},{"key":"ref_6","unstructured":"Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. (2023). QLoRA: Efficient Fine-tuning of Quantized LLMs. arXiv."},{"key":"ref_7","unstructured":"Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv."},{"key":"ref_8","unstructured":"Zhang, Q., Chen, M., Bukharin, A., Karampatziakis, N., He, P., Cheng, Y., Chen, W., and Zhao, T. (2023). AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning. arXiv."},{"key":"ref_9","unstructured":"Houlsby, N., Giurgiu, A., Jastrz\u0119bski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. (2019, January 9\u201315). Parameter-Efficient Transfer Learning for NLP. Proceedings of the 36th International Conference on Machine Learning (ICML 2019), Long Beach, CA, USA. Available online: https:\/\/proceedings.mlr.press\/v97\/houlsby19a.html."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ben Zaken, E., Ravfogel, S., and Goldberg, Y. (2021). BitFit: Simple Parameter-Efficient Fine-Tuning for Transformer-Based Masked Language Models. arXiv.","DOI":"10.18653\/v1\/2022.acl-short.1"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"R\u00fcckl\u00e9, A., Geigle, G., Glockner, M., Beck, T., Pfeiffer, J., Reimers, N., and Gurevych, I. (2021, January 7\u201311). AdapterDrop: On the Efficiency of Adapters in Transformers. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), Punta Cana, Dominican Republic.","DOI":"10.18653\/v1\/2021.emnlp-main.626"},{"key":"ref_12","unstructured":"Shulman, Y. (2020). DiffPrune: Neural Network Pruning with Deterministic Approximate Binary Gates and L0 Regularization. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1038\/s42256-023-00626-4","article-title":"Parameter-Efficient Fine-Tuning of Large-Scale Pre-Trained Language Models","volume":"5","author":"Ding","year":"2023","journal-title":"Nat. Mach. Intell."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"He, Y., Zhu, X., Li, D., and Wang, H. (2025). Enhancing Large Language Models for Specialized Domains: A Two-Stage Framework with Parameter-Sensitive LoRA Fine-Tuning and Chain-of-Thought RAG. Electronics, 14.","DOI":"10.3390\/electronics14101961"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Mao, Y., Ge, Y., Fan, Y., Xu, W., Mi, Y., Hu, Z., and Gao, Y. (2024). A Survey on LoRA of Large Language Models. Front. Comput. Sci., 19.","DOI":"10.1007\/s11704-024-40663-9"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"30667","DOI":"10.1038\/s41598-024-75599-4","article-title":"Parameter-Efficient Fine-Tuning of Large Language Models Using Semantic Knowledge Tuning","volume":"14","author":"Prottasha","year":"2024","journal-title":"Sci. Rep."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1007\/s10462-025-11236-4","article-title":"Parameter-Efficient Fine-Tuning in Large Language Models: A Survey of Methodologies","volume":"58","author":"Wang","year":"2025","journal-title":"Artif. Intell. Rev."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2385268","DOI":"10.1080\/08839514.2024.2385268","article-title":"A New Adapter Tuning of Large Language Model for Chinese Medical Named Entity Recognition","volume":"38","author":"Zhou","year":"2024","journal-title":"Appl. Artif. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wang, Y., Wang, J., and Zhang, X. (2023, January 13). YNU-HPCC at WASSA-2023 Shared Task 1: Large-Scale Language Model with LoRA Fine-Tuning for Empathy Detection and Emotion Classification. Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis (WASSA 2023), Toronto, Canada.","DOI":"10.18653\/v1\/2023.wassa-1.45"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1007\/s10618-022-00853-0","article-title":"Sentiment Analysis in Tweets: An Assessment Study from Classical to Modern Word Representation Models","volume":"37","author":"Barreto","year":"2023","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Shen, J.C., Su, N.J., and Lin, Y.B. (2025). Effective Multi-Class Sentiment Analysis Using Fine-Tuned Large Language Model with KNIME Analytics Platform. Systems, 13.","DOI":"10.3390\/systems13070523"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Dordevic, N., and Stojkovic, S. (2024, January 24\u201327). Traditional and Parameter-Efficient Fine-Tuning of LLMs for Sentiment Analysis in the English and Serbian Language. Proceedings of the 2024 11th International Conference on Electrical, Electronic and Computing Engineering (IcETRAN 2024), Ni\u0161, Serbia.","DOI":"10.1109\/IcETRAN62308.2024.10735424"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhan, T., Shi, C., Shi, Y., Li, H., and Lin, Y. (2024). Optimization Techniques for Sentiment Analysis Based on LLM (GPT-3). arXiv.","DOI":"10.54254\/2755-2721\/77\/2024MA0060"},{"key":"ref_24","unstructured":"Pavlyshenko, B.M. (2023). Analysis of Disinformation and Fake News Detection Using Fine-Tuned Large Language Model. arXiv."},{"key":"ref_25","unstructured":"Stanfordnlp\/Imdb (2025, December 08). Datasets at Hugging Face, Available online: https:\/\/huggingface.co\/datasets\/stanfordnlp\/imdb."},{"key":"ref_26","unstructured":"Stanfordnlp\/Sst2 (2025, December 08). Datasets at Hugging Face, Available online: https:\/\/huggingface.co\/datasets\/stanfordnlp\/sst2."},{"key":"ref_27","unstructured":"(2025, December 08). Yelp_Polarity_Reviews TensorFlow Datasets. Available online: https:\/\/www.tensorflow.org\/datasets\/catalog\/yelp_polarity_reviews."},{"key":"ref_28","unstructured":"Cardiffnlp\/Tweet_eval (2025, December 08). Datasets at Hugging Face, Available online: https:\/\/huggingface.co\/datasets\/cardiffnlp\/tweet_eval\/viewer\/sentiment."}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/15\/3\/159\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T10:15:42Z","timestamp":1772532942000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/15\/3\/159"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,3]]},"references-count":28,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["computers15030159"],"URL":"https:\/\/doi.org\/10.3390\/computers15030159","relation":{},"ISSN":["2073-431X"],"issn-type":[{"value":"2073-431X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,3]]}}}