{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:42:48Z","timestamp":1760060568800,"version":"build-2065373602"},"reference-count":40,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2025,9,1]],"date-time":"2025-09-01T00:00:00Z","timestamp":1756684800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>In today\u2019s competitive digital landscape, application usability plays a critical role in user satisfaction and retention. Negative user reviews offer valuable insights into real-world usability issues, yet traditional analysis methods often fall short in scalability and contextual understanding. This paper proposes an intelligent framework that utilizes large language models (LLMs), including GPT-4, Gemini, and BLOOM, to automate the extraction of actionable usability recommendations from negative app reviews. By applying prompting and fine-tuning techniques, the framework transforms unstructured feedback into meaningful suggestions aligned with three core usability dimensions: correctness, completeness, and satisfaction. A manually annotated dataset of Instagram negative reviews was used to evaluate model performance. Results show that GPT-4 consistently outperformed other models, achieving BLEU scores up to 0.64, ROUGE scores up to 0.80, and METEOR scores up to 0.90\u2014demonstrating high semantic accuracy and contextual relevance in generated recommendations. Gemini and BLOOM, while improved through fine-tuning, showed significantly lower performance. This study also introduces a practical, web-based tool that enables real-time review analysis and recommendation generation, supporting data-driven, user-centered software development. These findings illustrate the potential of LLM-based frameworks to enhance software usability analysis and accelerate feedback-driven design processes.<\/jats:p>","DOI":"10.3390\/computers14090363","type":"journal-article","created":{"date-parts":[[2025,9,2]],"date-time":"2025-09-02T08:23:38Z","timestamp":1756801418000},"page":"363","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Enhancing Software Usability Through LLMs: A Prompting and Fine-Tuning Framework for Analyzing Negative User Feedback"],"prefix":"10.3390","volume":"14","author":[{"given":"Nahed","family":"Alsaleh","sequence":"first","affiliation":[{"name":"Department of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia"},{"name":"Department of Computer Science, College of Computer Science and Engineering, University of Hail, Hail 55473, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Reem","family":"Alnanih","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4683-3716","authenticated-orcid":false,"given":"Nahed","family":"Alowidi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,9,1]]},"reference":[{"key":"ref_1","unstructured":"UXCam (2025, August 15). UX Statistics Every Designer Should Know. Available online: https:\/\/uxcam.com\/blog\/ux-statistics\/."},{"key":"ref_2","unstructured":"Testlio (2025, August 15). Mobile App Testing Statistics. Available online: https:\/\/testlio.com\/blog\/mobile-app-testing-statistics\/."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1007\/978-3-319-39510-4_25","article-title":"New ISO Standards for Usability, Usability Reports and Usability Measures","volume":"Volume 9731","author":"Kurosu","year":"2016","journal-title":"Human-Computer Interaction. Theory, Design, Development and Practice"},{"key":"ref_4","unstructured":"Asamoah, J. (2025). Evaluating Key Determinant of Customer Satisfaction: A Focus on Customer Relations, Service Quality, Product Quality and Supply Chain Management. [Master\u2019s Thesis, KAMK University of Applied Sciences]."},{"key":"ref_5","first-page":"363","article-title":"On Non-Functional Requirements in Software Engineering","volume":"Volume 5600","author":"Borgida","year":"2009","journal-title":"Conceptual Modeling: Foundations and Applications"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wang, T., Sudhir, K., and Hong, D. (2024). Using Advanced LLMs to Enhance Smaller LLMs: An Interpretable Knowledge Distillation Approach. arXiv.","DOI":"10.2139\/ssrn.4924881"},{"key":"ref_7","first-page":"27730","article-title":"Training language models to follow instructions with human feedback","volume":"35","author":"Ouyang","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_8","unstructured":"Liu, J., Liu, C., Zhou, P., Lv, R., Zhou, K., and Zhang, Y. (2023). Is ChatGPT a Good Recommender? A Preliminary Study. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Dai, S., Shao, N., Zhao, H., Yu, W., Si, Z., Xu, C., Sun, Z., Zhang, X., and Xu, J. (2023, January 14). Uncovering ChatGPT\u2019s Capabilities in Recommender Systems. Proceedings of the 17th ACM Conference on Recommender Systems, Singapore.","DOI":"10.1145\/3604915.3610646"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Sanner, S., Balog, K., Radlinski, F., Wedin, B., and Dixon, L. (2023, January 14). Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferences. Proceedings of the 17th ACM Conference on Recommender Systems, Singapore.","DOI":"10.1145\/3604915.3608845"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Sileo, D., Vossen, W., and Raymaekers, R. (2021). Zero-Shot Recommendation as Language Modeling. arXiv.","DOI":"10.1007\/978-3-030-99739-7_26"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Hou, Y., Zhang, J., Lin, Z., Lu, H., Xie, R., McAuley, J., and Zhao, W.X. (2024). Large Language Models are Zero-Shot Rankers for Recommender Systems. arXiv.","DOI":"10.1007\/978-3-031-56060-6_24"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Sun, W., Yan, L., Ma, X., Wang, S., Ren, P., Chen, Z., Yin, D., and Ren, Z. (2024). Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents. arXiv.","DOI":"10.18653\/v1\/2023.emnlp-main.923"},{"key":"ref_14","unstructured":"Yang, Z., Wu, J., Luo, Y., Zhang, J., Yuan, Y., Zhang, A., Wang, X., and He, X. (2023). Large Language Model Can Interpret Latent Space of Sequential Recommender. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yang, S., Ma, W., Sun, P., Ai, Q., Liu, Y., Cai, M., and Zhang, M. (2024, January 11). Sequential Recommendation with Latent Relations based on Large Language Model. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, Washington, DC, USA.","DOI":"10.1145\/3626772.3657762"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Lyu, H., Jiang, S., Zeng, H., Xia, Y., Wang, Q., Zhang, S., Chen, R., Leung, C., Tang, J., and Luo, J. (2024). LLM-Rec: Personalized Recommendation via Prompting Large Language Models. arXiv.","DOI":"10.18653\/v1\/2024.findings-naacl.39"},{"key":"ref_17","unstructured":"Wang, L., and Lim, E.-P. (2023). Zero-Shot Next-Item Recommendation using Large Pretrained Language Models. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wang, Y., Chu, Z., Ouyang, X., Wang, S., Hao, H., Shen, Y., Gu, J., Xue, S., Zhang, J.Y., and Cui, Q. (2024). Enhancing Recommender Systems with Large Language Model Reasoning Graphs. arXiv.","DOI":"10.1609\/aaai.v38i17.29887"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wei, W., Ren, X., Tang, J., Wang, Q., Su, L., Cheng, S., Wang, J., Yin, D., and Huang, C. (2024). LLMRec: Large Language Models with Graph Augmentation for Recommendation. arXiv.","DOI":"10.1145\/3616855.3635853"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ren, X., Wei, W., Xia, L., Su, L., Cheng, S., Wang, J., Yin, D., and Huang, C. (2024, January 13\u201317). Representation Learning with Large Language Models for Recommendation. Proceedings of the ACM Web Conference 2024, Singapore.","DOI":"10.1145\/3589334.3645458"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Liu, Q., Chen, N., Sakai, T., and Wu, X.-M. (2024, January 4\u20138). ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Models. Proceedings of the 17th ACM International Conference on Web Search and Data Mining, Merida, Mexico.","DOI":"10.1145\/3616855.3635845"},{"key":"ref_22","unstructured":"Gao, Y., Sheng, T., Xiang, Y., Xiong, Y., Wang, H., and Zhang, J. (2023). Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Shu, Y., Zhang, H., Gu, H., Zhang, P., Lu, T., Li, D., and Gu, N. (2023). RAH! RecSys-Assistant-Human: A Human-Centered Recommendation Framework with LLM Agents. arXiv.","DOI":"10.1109\/TCSS.2024.3404039"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Shi, W., He, X., Zhang, Y., Gao, C., Li, X., Zhang, J., Wang, Q., and Feng, F. (2024, January 14\u201318). Large Language Models are Learnable Planners for Long-Term Recommendation. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, Washington DC, USA.","DOI":"10.1145\/3626772.3657683"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Huang, X., Lian, J., Lei, Y., Yao, J., Lian, D., and Xie, X. (2024). Recommender AI Agent: Integrating Large Language Models for Interactive Recommendations. arXiv.","DOI":"10.1145\/3731446"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"103600","DOI":"10.1016\/j.jretconser.2023.103600","article-title":"Can chatbot customer service match human service agents on customer satisfaction? An investigation in the role of trust","volume":"76","author":"Huang","year":"2024","journal-title":"J. Retail. Consum. Serv."},{"key":"ref_27","unstructured":"Zhang, J., Bao, K., Wang, W., Zhang, Y., Shi, W., Xu, W., Feng, F., and Chua, T.-S. (2024). Prospect Personalized Recommendation on Large Language Model-based Agent Platform. arXiv."},{"key":"ref_28","unstructured":"Petrov, A.V., and Macdonald, C. (2023). Generative Sequential Recommendation with GPTRec. arXiv."},{"key":"ref_29","unstructured":"Wang, L., Zhang, J., Yang, H., Chen, Z., Tang, J., Zhang, Z., Chen, X., Lin, Y., Song, R., and Zhao, W.X. (2024). User Behavior Simulation with Large Language Model based Agents. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zhang, A., Chen, Y., Sheng, L., Wang, X., and Chua, T.-S. (2024, January 14\u201318). On Generative Agents in Recommendation. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, Washington, DC, USA.","DOI":"10.1145\/3626772.3657844"},{"key":"ref_31","unstructured":"Zhang, W., Wu, C., Li, X., Wang, Y., Dong, K., Wang, Y., Dai, X., Zhao, X., Guo, H., and Tang, R. (2024). LLMTreeRec: Unleashing the Power of Large Language Models for Cold-Start Recommendations. arXiv."},{"key":"ref_32","unstructured":"Erhan, D., Courville, A., Bengio, Y., and Vincent, P. (2010, January 13\u201315). Why Does Unsupervised Pre-training Help Deep Learning?. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Cagliari, Italy."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Kang, Y., Wang, Z., Zhang, H., Chen, J., and You, H. (2021, January 7\u201311). APIRecX: Cross-Library API Recommendation via Pre-Trained Language Model. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Punta Cana, Dominican Republic.","DOI":"10.18653\/v1\/2021.emnlp-main.275"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Wang, L., Hu, H., Sha, L., Xu, C., Wong, K.-F., and Jiang, D. (2022). RecInDial: A Unified Framework for Conversational Recommendation with Pretrained Language Models. arXiv.","DOI":"10.18653\/v1\/2022.aacl-main.37"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Xiao, S., Liu, Z., Shao, Y., Di, T., and Xie, X. (2021). Training Large-Scale News Recommenders with Pretrained Language Models in the Loop. arXiv.","DOI":"10.1145\/3534678.3539120"},{"key":"ref_36","unstructured":"Kang, W.-C., Ni, J., Mehta, N., Sathiamoorthy, M., Hong, L., Chi, E., and Cheng, D.Z. (2023). Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction. arXiv."},{"key":"ref_37","unstructured":"Li, R., Deng, W., Cheng, Y., Yuan, Z., Zhang, J., and Yuan, F. (2023). Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights. arXiv."},{"key":"ref_38","first-page":"949","article-title":"Hybrid Deep Learning Approach for Automating App Review Classification: Advancing Usability Metrics Classification with an Aspect-Based Sentiment Analysis Framework","volume":"82","author":"Alowidi","year":"2025","journal-title":"Comput. Mater. Contin."},{"key":"ref_39","unstructured":"Pfleeger, S.L. (1991). Software Engineering: The Production of Quality Software, Prentice Hall. [2nd ed.]."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1177\/1071181320641090","article-title":"ISO Human-Computer Interaction Standards: Finding Them and What They Contain","volume":"64","author":"Green","year":"2020","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet."}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/9\/363\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:37:00Z","timestamp":1760035020000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/9\/363"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,1]]},"references-count":40,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["computers14090363"],"URL":"https:\/\/doi.org\/10.3390\/computers14090363","relation":{},"ISSN":["2073-431X"],"issn-type":[{"type":"electronic","value":"2073-431X"}],"subject":[],"published":{"date-parts":[[2025,9,1]]}}}