{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T13:58:30Z","timestamp":1770818310840,"version":"3.50.1"},"reference-count":116,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T00:00:00Z","timestamp":1762041600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T00:00:00Z","timestamp":1762041600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100014597","name":"Universidade da Coru\u00f1a","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100014597","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2026,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Context<\/jats:title>\n                    <jats:p>In recent years, the multi-class classification of non-functional requirements has seen improvements through the use of Machine Learning algorithms. However, challenges such as data scarcity and class imbalance persist, particularly for languages other than English, such as Spanish.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aims to analyze the performance metrics of Machine Learning algorithms for classifying non-functional requirements translated into and originally written in Spanish. It evaluates the effectiveness of dataset balancing techniques and conducts cross-dataset validation to assess the generalizability of the models.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Method<\/jats:title>\n                    <jats:p>\n                      A dataset balancing process was conducted using a combination of oversampling and undersampling techniques. Six algorithms were trained in two experiments using a hyperparameter tuning process, employing two different datasets:\n                      <jats:italic>PROMISE_exp_translated<\/jats:italic>\n                      and the newly\n                      <jats:italic>PROMISE_exp_balanced<\/jats:italic>\n                      . The best-performing models were further tested on unseen data to evaluate their generalizability.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Logistic Regression and Naive Bayes demonstrated superior performance on the translated dataset, achieving f1-scores of 82% and 81%, respectively. Although overall performance decreased on the balanced dataset, specific underrepresented classes such as\n                      <jats:italic>Portability<\/jats:italic>\n                      and\n                      <jats:italic>Fault Tolerance<\/jats:italic>\n                      benefited from the balancing process.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Shallow Machine Learning algorithms are effective for classifying Spanish non-functional requirements, particularly when addressing data imbalance. The study highlights the importance of dataset balancing in improving classification performance for specific classes and provides insights into the challenges of generalizing models across datasets.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1007\/s10664-025-10736-9","type":"journal-article","created":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T21:43:07Z","timestamp":1762119787000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Improving the Multi-Class Classification of Non-Functional Requirements in Spanish: A Study of Dataset Balancing and Performance"],"prefix":"10.1007","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9619-924X","authenticated-orcid":false,"given":"M.","family":"Limaylla-Lunarejo","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1044-3871","authenticated-orcid":false,"given":"N.","family":"Condori-Fernandez","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0549-2000","authenticated-orcid":false,"given":"M.","family":"Rodr\u00edguez Luaces","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5336-6899","authenticated-orcid":false,"given":"O.","family":"Karras","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,11,2]]},"reference":[{"key":"10736_CR1","unstructured":"Iso\/iec 25010 (2011) 2011 systems and software engineering - systems and software quality requirements and evaluation (square) - system and software quality models"},{"key":"10736_CR2","doi-asserted-by":"publisher","unstructured":"Abad ZSH., Karras O, Ghazi P, Glinz M, Ruhe G, Schneider K (2017) What Works Better? A Study of Classifying Requirements. In: Proceedings - 2017 IEEE 25th International Requirements Engineering Conference, RE 2017. https:\/\/doi.org\/10.1109\/RE.2017.36","DOI":"10.1109\/RE.2017.36"},{"key":"10736_CR3","doi-asserted-by":"crossref","unstructured":"Abbas J, Zhang C, Luo B (2024) Enscl-catboost: A strategic framework for software requirements classification. IEEE Access","DOI":"10.1109\/ACCESS.2024.3452011"},{"key":"10736_CR4","doi-asserted-by":"crossref","unstructured":"Agarwal A, Mittal K, Doyle A, Sridhar P, Wan Z, Doughty JA, Savelka J, Sakr M (2024) Understanding the role of temperature in diverse question generation by gpt-4. In: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 2, pp. 1550\u20131551","DOI":"10.1145\/3626253.3635608"},{"key":"10736_CR5","doi-asserted-by":"crossref","unstructured":"Aggarwal CC (2018) Linear classification and regression for text. In: Machine Learning for Text, pp. 159\u2013207. Springer","DOI":"10.1007\/978-3-319-73531-3_6"},{"key":"10736_CR6","doi-asserted-by":"publisher","first-page":"8830683","DOI":"10.1155\/2020\/8830683","volume":"2020","author":"A Ahmad","year":"2020","unstructured":"Ahmad A, Feng C, Khan M, Khan A, Ullah A, Nazir S, Tahir A (2020) A systematic literature review on using machine learning algorithms for software requirements identification on stack overflow. Security and Communication Networks 2020:8830683","journal-title":"Security and Communication Networks"},{"key":"10736_CR7","doi-asserted-by":"crossref","unstructured":"Akkaradamrongrat S, Kachamas P, Sinthupinyo S (2019) Text generation for imbalanced text classification. In: 2019 16th International Joint Conference on Computer Science and Software Engineering (JCSSE), pp. 181\u2013186. IEEE","DOI":"10.1109\/JCSSE.2019.8864181"},{"key":"10736_CR8","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2022.102806","volume":"218","author":"AM Alashqar","year":"2022","unstructured":"Alashqar AM (2022) Studying the commonalities, mappings and relationships between non-functional requirements using machine learning. Sci Comput Program 218:102806","journal-title":"Sci Comput Program"},{"key":"10736_CR9","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2022.106877","volume":"147","author":"O AlDhafer","year":"2022","unstructured":"AlDhafer O, Ahmad I, Mahmood S (2022) An end-to-end deep learning system for requirements classification using recurrent neural networks. Inf Softw Technol 147:106877","journal-title":"Inf Softw Technol"},{"key":"10736_CR10","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2023.107202","volume":"159","author":"W Alhoshan","year":"2023","unstructured":"Alhoshan W, Ferrari A, Zhao L (2023) Zero-shot learning for requirements classification: an exploratory study. Inf Softw Technol 159:107202","journal-title":"Inf Softw Technol"},{"key":"10736_CR11","doi-asserted-by":"crossref","unstructured":"Alrumaih H, Mirza A, Alsalamah H (2018) Toward automated software requirements classification. In: 2018 21st Saudi Computer Society National Computer Conference (NCC), pp. 1\u20136. IEEE","DOI":"10.1109\/NCG.2018.8593012"},{"key":"10736_CR12","doi-asserted-by":"crossref","unstructured":"Apaza RDG, Barrios JEM, Becerra DAI, Quispe JAH (2018) Ers-tool: hybrid model for software requirements elicitation in spanish language. In: Proceedings of the International Conference on Geoinformatics and Data Analysis, pp. 27\u201330","DOI":"10.1145\/3220228.3220255"},{"issue":"1","key":"10736_CR13","doi-asserted-by":"publisher","first-page":"7240","DOI":"10.1038\/s41598-023-33607-z","volume":"13","author":"S Auer","year":"2023","unstructured":"Auer S, Barone DA, Bartz C, Cortes EG, Jaradeh MY, Karras O, Koubarakis M, Mouromtsev D, Pliukhin D, Radyush D et al (2023) The sciqa scientific question answering benchmark for scholarly knowledge. Sci Rep 13(1):7240","journal-title":"Sci Rep"},{"key":"10736_CR14","doi-asserted-by":"crossref","unstructured":"Baker C, Deng L, Chakraborty S, Dehlinger J (2019) Automatic multi-class non-functional software requirements classification using neural networks. In: 2019 IEEE 43rd annual computer software and applications conference (COMPSAC), vol.\u00a02, pp. 610\u2013615. IEEE","DOI":"10.1109\/COMPSAC.2019.10275"},{"issue":"7","key":"10736_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3544558","volume":"55","author":"M Bayer","year":"2022","unstructured":"Bayer M, Kaufhold MA, Reuter C (2022) A survey on data augmentation for text classification. ACM Comput Surv 55(7):1\u201339","journal-title":"ACM Comput Surv"},{"key":"10736_CR16","unstructured":"Bencheikh L, H\u00f6glund N (2023) Exploring the efficacy of chatgpt in generating requirements: An experimental study"},{"key":"10736_CR17","doi-asserted-by":"publisher","DOI":"10.7717\/peerj-cs.906","volume":"8","author":"JA Ben\u00edtez-Andrades","year":"2022","unstructured":"Ben\u00edtez-Andrades JA, Gonz\u00e1lez-Jim\u00e9nez \u00c1, L\u00f3pez-Brea \u00c1, Aveleira-Mata J, Alija-P\u00e9rez JM, Garc\u00eda-Ord\u00e1s MT (2022) Detecting racism and xenophobia using deep learning models on twitter data: cnn, lstm and bert. PeerJ Computer Science 8:e906","journal-title":"PeerJ Computer Science"},{"issue":"6","key":"10736_CR18","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1007\/s10664-021-09986-0","volume":"26","author":"DM Berry","year":"2021","unstructured":"Berry DM (2021) Empirical evaluation of tools for hairy requirements engineering tasks. Empir Softw Eng 26(6):111","journal-title":"Empir Softw Eng"},{"key":"10736_CR19","volume":"1","author":"M Binkhonain","year":"2019","unstructured":"Binkhonain M, Zhao L (2019) A review of machine learning algorithms for identification and classification of non-functional requirements. Expert Syst Appl X 1:100001","journal-title":"Expert Syst Appl X"},{"key":"10736_CR20","volume-title":"Pattern recognition and machine learning","author":"CM Bishop","year":"2006","unstructured":"Bishop CM, Nasrabadi NM (2006) Pattern recognition and machine learning, vol 4. Springer, Berlin"},{"key":"10736_CR21","first-page":"1877","volume":"33","author":"T Brown","year":"2020","unstructured":"Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877\u20131901","journal-title":"Adv Neural Inf Process Syst"},{"key":"10736_CR22","unstructured":"Ca\u00f1ete J, Chaperon G, Fuentes R, Ho JH, Kang H, P\u00e9rez J (2020) Spanish pre-trained bert model and evaluation data. In: PML4DC at ICLR 2020"},{"key":"10736_CR23","unstructured":"Cardellino C (2019) Spanish Billion Words Corpus and Embeddings. https:\/\/crscardellino.github.io\/SBWCE\/"},{"key":"10736_CR24","doi-asserted-by":"crossref","unstructured":"Chatterjee R, Ahmed A, Anish PR, Suman B, Lawhatre P, Ghaisas S (2021) A pipeline for automating labeling to prediction in classification of nfrs. In: 2021 IEEE 29th International Requirements Engineering Conference (RE), pp. 323\u2013323. IEEE","DOI":"10.1109\/RE51729.2021.00036"},{"key":"10736_CR25","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1613\/jair.953","volume":"16","author":"NV Chawla","year":"2002","unstructured":"Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research 16:321\u2013357","journal-title":"Journal of artificial intelligence research"},{"issue":"3\/4","key":"10736_CR26","doi-asserted-by":"publisher","first-page":"256","DOI":"10.2307\/2332378","volume":"37","author":"WG Cochran","year":"1950","unstructured":"Cochran WG (1950) The comparison of percentages in matched samples. Biometrika 37(3\/4):256\u2013266","journal-title":"Biometrika"},{"key":"10736_CR27","unstructured":"Copae DV (2022) Using augmented software requirements for automatic classification. B.S. thesis, University of Twente"},{"key":"10736_CR28","unstructured":"Cruciani F, Moore S, Nugent C (2023) Comparing general purpose pre-trained word and sentence embeddings forrequirements classification. In: 6th Workshop on Natural Language Processing for Requirements Engineering: REFSQ Co-Located Events 2023. CEUR-WS"},{"key":"10736_CR29","unstructured":"Dai H, Liu Z, Liao W, Huang X, Wu Z, Zhao L, Liu W, Liu N, Li S, Zhu D et\u00a0al (2023) Chataug: Leveraging chatgpt for text data augmentation. arXiv:2302.13007"},{"issue":"2","key":"10736_CR30","doi-asserted-by":"publisher","first-page":"37","DOI":"10.5120\/3358-4633","volume":"28","author":"MK Dalal","year":"2011","unstructured":"Dalal MK, Zaveri MA (2011) Automatic text classification: a technical review. International Journal of Computer Applications 28(2):37\u201340","journal-title":"International Journal of Computer Applications"},{"key":"10736_CR31","doi-asserted-by":"crossref","unstructured":"Dalpiaz F, Dell\u2019Anna D, Aydemir FB, \u00c7evikol S (2019) Requirements classification with interpretable machine learning and dependency parsing. In: 2019 IEEE 27th International Requirements Engineering Conference (RE), pp. 142\u2013152. IEEE","DOI":"10.1109\/RE.2019.00025"},{"key":"10736_CR32","unstructured":"de\u00a0Paula AFM, da\u00a0Silva RF, Schlicht IB (2021) Sexism prediction in spanish and english tweets using monolingual and multilingual bert and ensemble models. arXiv:2111.04551"},{"key":"10736_CR33","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805"},{"issue":"9","key":"10736_CR34","doi-asserted-by":"publisher","first-page":"1057","DOI":"10.3390\/e22091057","volume":"22","author":"E Dias Canedo","year":"2020","unstructured":"Dias Canedo E, Cordeiro Mendes B (2020) Software requirements classification using machine learning algorithms. Entropy 22(9):1057","journal-title":"Entropy"},{"key":"10736_CR35","doi-asserted-by":"crossref","unstructured":"Fan A, Gokkaya B, Harman M, Lyubarskiy M, Sengupta S, Yoo S, Zhang JM (2023) Large language models for software engineering: Survey and open problems. arXiv:2310.03533","DOI":"10.1109\/ICSE-FoSE59343.2023.00008"},{"key":"10736_CR36","unstructured":"Fong V (2018) Software requirements classification using word embeddings and convolutional neural networks. Master\u2019s thesis, California Polytechnic State University"},{"issue":"8","key":"10736_CR37","doi-asserted-by":"publisher","first-page":"705","DOI":"10.1134\/S0361768823080133","volume":"49","author":"SM Garc\u00eda","year":"2023","unstructured":"Garc\u00eda SM, Fern\u00e1ndez-y Fern\u00e1ndez CA, P\u00e9rez ER (2023) Classification of non-functional requirements using convolutional neural networks. Program Comput Softw 49(8):705\u2013711","journal-title":"Program Comput Softw"},{"key":"10736_CR38","first-page":"9671","volume-title":"Herramienta de pnl para la detecci\u00f3n de ambig\u00fcedades en requisitos de software escritos en espa\u00f1ol nlp tool for the detection of ambiguities in software requirements written in spanish","author":"SE Gonz\u00e1lez","year":"2023","unstructured":"Gonz\u00e1lez SE, Reyes JR, Cede\u00f1o DC, Azcuy RA, Diez HG (2023) Herramienta de pnl para la detecci\u00f3n de ambig\u00fcedades en requisitos de software escritos en espa\u00f1ol nlp tool for the detection of ambiguities in software requirements written in spanish. V Convenci\u00f3n Cient\u00edfica Internacional Uciencia, Varadero, Cuba, Tech Rep, p 9671"},{"key":"10736_CR39","doi-asserted-by":"crossref","unstructured":"Haque MA, Rahman MA, Siddik MS (2019) Non-functional requirements classification with feature extraction and machine learning: An empirical study. In: 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT), pp. 1\u20135. IEEE","DOI":"10.1109\/ICASERT.2019.8934499"},{"issue":"9","key":"10736_CR40","doi-asserted-by":"publisher","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","volume":"21","author":"H He","year":"2009","unstructured":"He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263\u20131284. https:\/\/doi.org\/10.1109\/TKDE.2008.239","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"10736_CR41","doi-asserted-by":"publisher","unstructured":"Henriquez Miranda C, Guzman J (2016) A review of Sentiment Analysis in Spanish. TECCIENCIA 12(22):35\u201348.\u00a0https:\/\/doi.org\/10.18180\/tecciencia.2017.22.5","DOI":"10.18180\/tecciencia.2017.22.5"},{"key":"10736_CR42","doi-asserted-by":"crossref","unstructured":"Hey T, Keim J, Koziolek A, Tichy WF (2020) Norbert: Transfer learning for requirements classification. In: 2020 IEEE 28th International Requirements Engineering Conference (RE), pp. 169\u2013179. IEEE","DOI":"10.1109\/RE48521.2020.00028"},{"key":"10736_CR43","unstructured":"Hmeidi II, Khaleel MI, Najadat HM (2016) Automatic text classification: A comparative study. In: Proceedings of the International Computer Sciences and Informatics Conference (ICSIC 2016), pp. 179\u2013189"},{"key":"10736_CR44","unstructured":"Hollander M (2013) Nonparametric statistical methods. Hoboken, John Wiley & Sons Inc"},{"issue":"2","key":"10736_CR45","first-page":"1","volume":"5","author":"M Hossin","year":"2015","unstructured":"Hossin M, Sulaiman MN (2015) A review on evaluation metrics for data classification evaluations. International journal of data mining & knowledge management process 5(2):1","journal-title":"International journal of data mining & knowledge management process"},{"key":"10736_CR46","unstructured":"Instituto Cervantes (2022) El espa\u00f1ol una lengua viva. https:\/\/cvc.cervantes.es\/lengua\/espanol_lengua_viva\/pdf\/espanol_lengua_viva_2022.pdf. [Online; accessed 25-April-2023]"},{"key":"10736_CR47","unstructured":"Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp. 448\u2013456. PMLR"},{"issue":"3","key":"10736_CR48","doi-asserted-by":"publisher","first-page":"685","DOI":"10.1007\/s12525-021-00475-2","volume":"31","author":"C Janiesch","year":"2021","unstructured":"Janiesch C, Zschech P, Heinrich K (2021) Machine learning and deep learning. Electron Mark 31(3):685\u2013695","journal-title":"Electron Mark"},{"key":"10736_CR49","doi-asserted-by":"publisher","DOI":"10.1017\/9781009003872","volume-title":"Machine learning evaluation: towards reliable and responsible AI","author":"N Japkowicz","year":"2024","unstructured":"Japkowicz N, Boukouvalas Z (2024) Machine learning evaluation: towards reliable and responsible AI. Cambridge University Press, Cambridge"},{"key":"10736_CR50","doi-asserted-by":"crossref","unstructured":"Joulin A, Grave E, Bojanowski P, Mikolov T (2016) Bag of tricks for efficient text classification. arXiv:1607.01759","DOI":"10.18653\/v1\/E17-2068"},{"issue":"1","key":"10736_CR51","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1007\/s10462-018-09677-1","volume":"52","author":"AI Kadhim","year":"2019","unstructured":"Kadhim AI (2019) Survey on supervised machine learning techniques for automatic text classification. Artif Intell Rev 52(1):273\u2013292","journal-title":"Artif Intell Rev"},{"key":"10736_CR52","doi-asserted-by":"publisher","DOI":"10.1016\/j.compeleceng.2023.108699","volume":"108","author":"K Kaur","year":"2023","unstructured":"Kaur K, Kaur P (2023) Improving bert model for requirements classification by bidirectional lstm-cnn deep model. Comput Electr Eng 108:108699","journal-title":"Comput Electr Eng"},{"issue":"3","key":"10736_CR53","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1007\/s10462-023-10667-1","volume":"57","author":"K Kaur","year":"2024","unstructured":"Kaur K, Kaur P (2024) The application of ai techniques in requirements classification: a systematic mapping. Artif Intell Rev 57(3):57","journal-title":"Artif Intell Rev"},{"key":"10736_CR54","doi-asserted-by":"crossref","unstructured":"Kavitha M, Prabhavathy P (2021) A review on machine learning techniques for text classification. In: 2021 4th International Conference on Computing and Communications Technologies (ICCCT), pp. 605\u2013610. IEEE","DOI":"10.1109\/ICCCT53315.2021.9711858"},{"key":"10736_CR55","doi-asserted-by":"crossref","unstructured":"Khan MA, Khan MS, Khan I, Ahmad S, Huda S (2023) Non functional requirements identification and classification using transfer learning model. IEEE Access","DOI":"10.1109\/ACCESS.2023.3295238"},{"key":"10736_CR56","doi-asserted-by":"crossref","unstructured":"Khatian VM, Arain QA, Alenezi M, Raza MO, Shaikh F, Farah I (2021) Comparative analysis for predicting non-functional requirements using supervised machine learning. In: 2021 1st International Conference on Artificial Intelligence and Data Analytics (CAIDA), pp. 7\u201312. IEEE","DOI":"10.1109\/CAIDA51941.2021.9425236"},{"key":"10736_CR57","doi-asserted-by":"publisher","unstructured":"Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746\u20131751. Association for Computational Linguistics, Doha, Qatar https:\/\/doi.org\/10.3115\/v1\/D14-1181","DOI":"10.3115\/v1\/D14-1181"},{"key":"10736_CR58","doi-asserted-by":"crossref","unstructured":"Kumar V, Choudhary A, Cho E (2020) Data augmentation using pre-trained transformer models. arXiv:2003.02245","DOI":"10.18653\/v1\/2020.lifelongnlp-1.3"},{"key":"10736_CR59","doi-asserted-by":"publisher","unstructured":"Kurtanovic Z, Maalej W (2017) Automatically Classifying Functional and Non-functional Requirements Using Supervised Machine Learning. In: Proceedings - 2017 IEEE 25th International Requirements Engineering Conference, RE 2017 https:\/\/doi.org\/10.1109\/RE.2017.82","DOI":"10.1109\/RE.2017.82"},{"key":"10736_CR60","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2022.106991","volume":"150","author":"P Leelaprute","year":"2022","unstructured":"Leelaprute P, Amasaki S (2022) A comparative study on vectorization methods for non-functional requirements classification. Inf Softw Technol 150:106991","journal-title":"Inf Softw Technol"},{"key":"10736_CR61","doi-asserted-by":"crossref","unstructured":"Li B, Nong X (2022) Automatically classifying non-functional requirements using deep neural network. Pattern Recognition p. 108948","DOI":"10.1016\/j.patcog.2022.108948"},{"key":"10736_CR62","doi-asserted-by":"publisher","first-page":"30080","DOI":"10.1109\/ACCESS.2022.3159238","volume":"10","author":"G Li","year":"2022","unstructured":"Li G, Zheng C, Li M, Wang H (2022) Automatic requirements classification based on graph attention network. IEEE Access 10:30080\u201330090","journal-title":"IEEE Access"},{"key":"10736_CR63","doi-asserted-by":"crossref","unstructured":"Li LF, Jin-An NC, Kasirun ZM, Chua YP (2019) An empirical comparison of machine learning algorithms for classification of software requirements. International Journal of Advanced Computer Science and Applications 10(11)","DOI":"10.14569\/IJACSA.2019.0101135"},{"key":"10736_CR64","unstructured":"Li Q, Peng H, Li J, Xia C, Yang R, Sun L, Yu PS, He L (2020) A survey on text classification: From shallow to deep learning. arXiv:2008.00364"},{"key":"10736_CR65","doi-asserted-by":"crossref","unstructured":"Lima M, Valle V, Costa E, Lira F, Gadelha B (2019) Software engineering repositories: expanding the promise database. In: Proceedings of the XXXIII Brazilian Symposium on Software Engineering, pp. 427\u2013436","DOI":"10.1145\/3350768.3350776"},{"key":"10736_CR66","doi-asserted-by":"crossref","unstructured":"Limaylla-Lunarejo MI, Condori-Fernandez N, Luaces MR (2022) Towards an automatic requirements classification in a new spanish dataset. (accepted to be published). In: 2022 IEEE 30th International Requirements Engineering Conference (RE). IEEE","DOI":"10.1109\/RE54965.2022.00039"},{"key":"10736_CR67","doi-asserted-by":"crossref","unstructured":"Limaylla-Lunarejo MI, Condori-Fernandez N, Luaces MR (2023) Evaluating non-functional requirements classification for spanish text: Traditional vs. deep learning approaches. In: Neural Information Processing Systems Conference: LatinX in AI (LXAI) Research Workshop 2023","DOI":"10.52591\/lxai202312103"},{"key":"10736_CR68","doi-asserted-by":"crossref","unstructured":"Limaylla-Lunarejo MI, Condori-Fernandez N, Luaces MR (2023) Requirements classification using fasttext and beto in spanish documents. In: Requirements Engineering: Foundation for Software Quality: 29th International Working Conference, REFSQ 2023, Barcelona, Spain, April 17\u201320, 2023, Proceedings, pp. 159\u2013176. Springer","DOI":"10.1007\/978-3-031-29786-1_11"},{"key":"10736_CR69","doi-asserted-by":"publisher","unstructured":"Limaylla-Lunarejo MI, Condori-Fernandez N, Luaces MR (2023) Towards a fair dataset for non-functional requirements. In: Proceedings of the 38th ACM\/SIGAPP Symposium on Applied Computing, SAC \u201923, p. 1414\u20131421. Association for Computing Machinery, New York, NY, USA https:\/\/doi.org\/10.1145\/3555776.3578611","DOI":"10.1145\/3555776.3578611"},{"key":"10736_CR70","doi-asserted-by":"crossref","unstructured":"Liu P, Wang X, Xiang C, Meng W (2020) A survey of text data augmentation. In: 2020 International Conference on Computer Communication and Network Security (CCNS), pp. 191\u2013195. IEEE","DOI":"10.1109\/CCNS50731.2020.00049"},{"key":"10736_CR71","doi-asserted-by":"crossref","unstructured":"L\u00f3pez-Hern\u00e1ndez DA, Ochar\u00e1n-Hern\u00e1ndez JO, Mezura-Montes E, S\u00e1nchez-Garc\u00eda \u00c1J (2021) Automatic classification of software requirements using artificial neural networks: A systematic literature review. In: 2021 9th International Conference in Software Engineering Research and Innovation (CONISOFT), pp. 152\u2013160. IEEE","DOI":"10.1109\/CONISOFT52520.2021.00030"},{"key":"10736_CR72","doi-asserted-by":"publisher","unstructured":"Loya M, Sinha D, Futrell R (2023) Exploring the sensitivity of LLMs\u2019 decision-making capabilities: Insights from prompt variations and hyperparameters. In: H.\u00a0Bouamor, J.\u00a0Pino, K.\u00a0Bali (eds.) Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 3711\u20133716. Association for Computational Linguistics, Singapore. https:\/\/doi.org\/10.18653\/v1\/2023.findings-emnlp.241. https:\/\/aclanthology.org\/2023.findings-emnlp.241\/","DOI":"10.18653\/v1\/2023.findings-emnlp.241"},{"key":"10736_CR73","doi-asserted-by":"crossref","unstructured":"Lu M, Liang P (2017) Automatic classification of non-functional requirements from augmented app user reviews. In: Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering, pp. 344\u2013353","DOI":"10.1145\/3084226.3084241"},{"key":"10736_CR74","doi-asserted-by":"crossref","unstructured":"Luo X, Xue Y, Xing Z, Sun J (2022) Prcbert: Prompt learning for requirement classification using bert-based pretrained language models. In: Proceedings of the 37th IEEE\/ACM International Conference on Automated Software Engineering, pp. 1\u201313","DOI":"10.1145\/3551349.3560417"},{"key":"10736_CR75","doi-asserted-by":"crossref","unstructured":"Magalh\u00e3es C, Araujo J, Sardinha A (2021) Mare: an active learning approach for requirements classification. In: 2021 IEEE 29th International Requirements Engineering Conference (RE), pp. 516\u2013521. IEEE","DOI":"10.1109\/RE51729.2021.9714537"},{"key":"10736_CR76","doi-asserted-by":"crossref","unstructured":"Mahmoudi L, Salem M (2022) Improving multi-class text classification using balancing techniques. In: International Conference on Artificial Intelligence: Theories and Applications, pp. 264\u2013275. Springer","DOI":"10.1007\/978-3-031-28540-0_21"},{"key":"10736_CR77","doi-asserted-by":"publisher","unstructured":"Marques N, Silva RR, Bernardino J (2024) Using chatgpt in software requirements engineering: a comprehensive review. Future Internet 16(6):180. https:\/\/doi.org\/10.3390\/fi16060180. https:\/\/www.mdpi.com\/1999-5903\/16\/6\/180","DOI":"10.3390\/fi16060180"},{"issue":"3","key":"10736_CR78","doi-asserted-by":"publisher","first-page":"1361","DOI":"10.13053\/cys-28-3-4212","volume":"28","author":"GA Mat\u00edas-Mendoza","year":"2024","unstructured":"Mat\u00edas-Mendoza GA, Ledeneva Y, Garc\u00eda-Hern\u00e1ndez RA (2024) Spanish automatic text summarization: a survey. Computaci\u00f3n y Sistemas 28(3):1361\u20131376","journal-title":"Computaci\u00f3n y Sistemas"},{"key":"10736_CR79","unstructured":"McGuinness DL, Van\u00a0Harmelen F et\u00a0al (2004) Owl web ontology language overview. W3C Recommendation 10(10):2004"},{"key":"10736_CR80","doi-asserted-by":"crossref","unstructured":"Muhamad FPB, Mulyani E, Bunga MS, Mushafa AF (2023) Class balancing methods comparison for software requirements classification on support vector machines. Sinkron: Jurnal Dan Penelitian Teknik Informatika 8(2):1196\u20131208","DOI":"10.33395\/sinkron.v8i2.12415"},{"issue":"4","key":"10736_CR81","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1145\/2757001.2757003","volume":"1","author":"MA Musen","year":"2015","unstructured":"Musen MA (2015) The prot\u00e9g\u00e9 project: a look back and a look forward. AI matters 1(4):4\u201312","journal-title":"AI matters"},{"key":"10736_CR82","doi-asserted-by":"crossref","unstructured":"Navarro-Almanza R, Juarez-Ramirez R, Licea G (2017) Towards supporting software engineering using deep learning: A case of software requirements classification. In: 2017 5th International Conference in Software Engineering Research and Innovation (CONISOFT), pp. 116\u2013120. IEEE","DOI":"10.1109\/CONISOFT.2017.00021"},{"key":"10736_CR83","unstructured":"OpenAI (2023) Gpt-4 technical report. ArXiv abs\/2303.08774. https:\/\/api.semanticscholar.org\/CorpusID:257532815"},{"key":"10736_CR84","unstructured":"Opitz J, Burst S (2019) Macro f1 and macro f1. arXiv preprint arXiv:1911.03347"},{"key":"10736_CR85","doi-asserted-by":"crossref","unstructured":"Orellana G, Arias B, Orellana M, Saquicela V, Baculima F, Piedra N (2018) A study on the impact of pre-processing techniques in spanish and english text classification over short and large text documents. In: 2018 international conference on information systems and computer science (INCISCOS), pp. 277\u2013283. IEEE","DOI":"10.1109\/INCISCOS.2018.00047"},{"key":"10736_CR86","doi-asserted-by":"publisher","first-page":"736","DOI":"10.1016\/j.procs.2019.09.229","volume":"159","author":"C Padurariu","year":"2019","unstructured":"Padurariu C, Breaban ME (2019) Dealing with data imbalance in text classification. Procedia Computer Science 159:736\u2013745","journal-title":"Procedia Computer Science"},{"key":"10736_CR87","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825\u20132830","journal-title":"J Mach Learn Res"},{"key":"10736_CR88","doi-asserted-by":"crossref","unstructured":"Peer J, Mordecai Y, Reich Y (2024) Nlp4ref: Requirements classification and forecasting: From model-based design to large language models. In: 2024 IEEE Aerospace Conference, pp. 1\u201316. IEEE","DOI":"10.1109\/AERO58975.2024.10521022"},{"key":"10736_CR89","doi-asserted-by":"crossref","unstructured":"Queiroz\u00a0Abonizio H, Barbon\u00a0Junior S (2020) Pre-trained data augmentation for text classification. In: Brazilian Conference on Intelligent Systems, pp. 551\u2013565. Springer","DOI":"10.1007\/978-3-030-61377-8_38"},{"issue":"10","key":"10736_CR90","doi-asserted-by":"publisher","first-page":"1264","DOI":"10.3390\/e23101264","volume":"23","author":"N Rahimi","year":"2021","unstructured":"Rahimi N, Eassa F, Elrefaei L (2021) One-and two-phase software requirement classification using ensemble deep learning. Entropy 23(10):1264","journal-title":"Entropy"},{"key":"10736_CR91","doi-asserted-by":"crossref","unstructured":"Rahman K, Ghani A, Ahmad R, Sajjad SH (2023) Hybrid deep learning approach for nonfunctional software requirements classifications. In: 2023 International Conference on Communication, Computing and Digital Systems (C-CODE), pp. 1\u20135. IEEE","DOI":"10.1109\/C-CODE58145.2023.10139907"},{"key":"10736_CR92","doi-asserted-by":"publisher","first-page":"81787","DOI":"10.1109\/ACCESS.2023.3301725","volume":"11","author":"K Rahman","year":"2023","unstructured":"Rahman K, Ghani A, Alzahrani A, Tariq MU, Rahman AU (2023) Pre-trained model-based nfr classification: overcoming limited data challenges. IEEE Access 11:81787\u201381802. https:\/\/doi.org\/10.1109\/ACCESS.2023.3301725","journal-title":"IEEE Access"},{"key":"10736_CR93","doi-asserted-by":"crossref","unstructured":"Rahman MA, Haque MA, Tawhid MNA, Siddik MS (2019) Classifying non-functional requirements using rnn variants for quality software development. In: Proceedings of the 3rd ACM SIGSOFT International Workshop on Machine Learning Techniques for Software Quality Evaluation, pp. 25\u201330","DOI":"10.1145\/3340482.3342745"},{"key":"10736_CR94","doi-asserted-by":"publisher","first-page":"56","DOI":"10.5815\/ijisa.2023.03.05","volume":"15","author":"MA Rahman","year":"2023","unstructured":"Rahman MA, Nayem A, Siddik S (2023) Non-functional requirements classification using machine learning algorithms. International Journal of Intelligent Systems and Applications 15:56\u201369. https:\/\/doi.org\/10.5815\/ijisa.2023.03.05","journal-title":"International Journal of Intelligent Systems and Applications"},{"issue":"24","key":"10736_CR95","doi-asserted-by":"publisher","first-page":"638","DOI":"10.21105\/joss.00638","volume":"3","author":"S Raschka","year":"2018","unstructured":"Raschka S (2018) Mlxtend: providing machine learning and data science utilities and extensions to python\u2019s scientific computing stack. Journal of open source software 3(24):638","journal-title":"Journal of open source software"},{"key":"10736_CR96","unstructured":"Raschka S (2018) Model evaluation, model selection, and algorithm selection in machine learning. arXiv:1811.12808"},{"key":"10736_CR97","unstructured":"\u0158eh\u016f\u0159ek R, Sojka P (2010) Software Framework for Topic Modelling with Large Corpora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pp. 45\u201350. ELRA, Valletta, Malta. http:\/\/is.muni.cz\/publication\/884893\/en"},{"key":"10736_CR98","doi-asserted-by":"publisher","unstructured":"Romero-Sandoval M, Calder\u00f3n-Ram\u00edrez S, Sol\u00eds M (2023) Using gpt-3 as a text data augmentator for a complex text detector. In: 2023 IEEE 5th International Conference on BioInspired Processing (BIP), pp. 1\u20136 https:\/\/doi.org\/10.1109\/BIP60195.2023.10379347","DOI":"10.1109\/BIP60195.2023.10379347"},{"key":"10736_CR99","doi-asserted-by":"publisher","unstructured":"Silva B, Nascimento R, Rivero L, Braz G, Santos R, Martins L, Viana D (2024) Promise+: expandindo a base de dados de requisitos de software promise_exp. In: Anais do XXXVIII Simp\u00f3sio Brasileiro de Engenharia de Software, pp. 291\u2013301. SBC, Porto Alegre, RS, Brasil https:\/\/doi.org\/10.5753\/sbes.2024.3427. https:\/\/sol.sbc.org.br\/index.php\/sbes\/article\/view\/30370","DOI":"10.5753\/sbes.2024.3427"},{"key":"10736_CR100","doi-asserted-by":"crossref","unstructured":"Silva EJ, Zanchettin C (2016) On validation setup for multiclass imbalanced data sets. In: 2016 5th Brazilian Conference on Intelligent Systems (BRACIS), pp. 468\u2013473. IEEE","DOI":"10.1109\/BRACIS.2016.090"},{"key":"10736_CR101","doi-asserted-by":"crossref","unstructured":"Stapor K (2017) Evaluation of classifiers: current methods and future research directions. In: FedCSIS (Position Papers), pp. 37\u201340","DOI":"10.15439\/2017F530"},{"issue":"3","key":"10736_CR102","doi-asserted-by":"publisher","first-page":"279","DOI":"10.3390\/aerospace10030279","volume":"10","author":"A Tikayat Ray","year":"2023","unstructured":"Tikayat Ray A, Cole BF, Pinon Fischer OJ, White RT, Mavris DN (2023) Aerobertclassifier: classification of aerospace requirements using bert. Aerospace 10(3):279","journal-title":"Aerospace"},{"key":"10736_CR103","doi-asserted-by":"crossref","unstructured":"Tiun S, Mokhtar U, Bakar S, Saad S (2020) Classification of functional and non-functional requirement in software requirement using word2vec and fast text. In: journal of Physics: conference series, vol. 1529, p. 042077. IOP Publishing","DOI":"10.1088\/1742-6596\/1529\/4\/042077"},{"key":"10736_CR104","doi-asserted-by":"crossref","unstructured":"T\u00f3th L, Vid\u00e1cs L (2018a) Study of various classifiers for identification and classification of non-functional requirements. In: Computational Science and Its Applications\u2013ICCSA 2018: 18th International Conference, Melbourne, VIC, Australia, July 2-5, 2018, Proceedings, Part V 18, pp. 492\u2013503. Springer","DOI":"10.1007\/978-3-319-95174-4_39"},{"key":"10736_CR105","doi-asserted-by":"crossref","unstructured":"T\u00f3th L, Vid\u00e1cs L (2018b) Study of various classifiers for identification and classification of non-functional requirements. In: International Conference on Computational Science and Its Applications, pp. 492\u2013503. Springer","DOI":"10.1007\/978-3-319-95174-4_39"},{"key":"10736_CR106","unstructured":"Touvron H, Lavril T, Izacard G, Martinet X, Lachaux MA, Lacroix T, Rozi\u00e8re B, Goyal N, Hambro E, Azhar F et\u00a0al (2023) Llama: Open and efficient foundation language models. arXiv:2302.13971"},{"key":"10736_CR107","unstructured":"Ubani S, Polat SO, Nielsen R (2023) Zeroshotdataaug: Generating and augmenting training data with chatgpt. arXiv:2304.14334"},{"key":"10736_CR108","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser \u0141, Polosukhin I (2017) Attention is all you need. Advances in neural information processing systems 30"},{"key":"10736_CR109","doi-asserted-by":"crossref","unstructured":"Vijayvargiya S, Kumar L, Malapati A, Murthy LB, Misra S (2022) Software functional requirements classification using ensemble learning. In: International Conference on Computational Science and Its Applications, pp. 678\u2013691. Springer","DOI":"10.1007\/978-3-031-10548-7_49"},{"key":"10736_CR110","doi-asserted-by":"crossref","unstructured":"Vijayvargiya S, Kumar L, Murthy LB, Misra S (2022) Software requirements classification using deep-learning approach with various hidden layers. In: 2022 17th Conference on Computer Science and Intelligence Systems (FedCSIS), pp. 895\u2013904. IEEE","DOI":"10.15439\/2022F140"},{"key":"10736_CR111","doi-asserted-by":"publisher","unstructured":"White J, Hays S, Fu Q, Spencer-Smith J, Schmidt DC (2024) ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design, pp. 71\u2013108. Springer Nature Switzerland, Cham https:\/\/doi.org\/10.1007\/978-3-031-55642-5_4. https:\/\/doi.org\/10.1007\/978-3-031-55642-5_4","DOI":"10.1007\/978-3-031-55642-5_4"},{"issue":"1","key":"10736_CR112","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/sdata.2016.18","volume":"3","author":"MD Wilkinson","year":"2016","unstructured":"Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, Blomberg N, Boiten JW, da Silva Santos LB, Bourne PE et al (2016) The fair guiding principles for scientific data management and stewardship. Scientific data 3(1):1\u20139","journal-title":"Scientific data"},{"key":"10736_CR113","doi-asserted-by":"publisher","unstructured":"Wohlin C (2014) Guidelines for snowballing in systematic literature studies and a replication in software engineering. In: ACM Int. Conf. Proc. Series https:\/\/doi.org\/10.1145\/2601248.2601268","DOI":"10.1145\/2601248.2601268"},{"key":"10736_CR114","doi-asserted-by":"crossref","unstructured":"Yoo KM, Park D, Kang J, Lee SW, Park W (2021) Gpt3mix: Leveraging large-scale language models for text augmentation. arXiv:2104.08826","DOI":"10.18653\/v1\/2021.findings-emnlp.192"},{"key":"10736_CR115","unstructured":"Zhang X, LeCun Y (2015) Text understanding from scratch. arXiv:1502.01710"},{"key":"10736_CR116","unstructured":"Zhao WX, Zhou K, Li J, Tang T, Wang X, Hou Y, Min Y, Zhang B, Zhang J, Dong Z et\u00a0al (2023) A survey of large language models. arXiv:2303.18223"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-025-10736-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-025-10736-9","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-025-10736-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T11:25:24Z","timestamp":1770809124000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-025-10736-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,2]]},"references-count":116,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1]]}},"alternative-id":["10736"],"URL":"https:\/\/doi.org\/10.1007\/s10664-025-10736-9","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,2]]},"assertion":[{"value":"28 March 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 September 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 November 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}},{"value":"Not applicable","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}},{"value":"The authors declare that they have no conflict of interest.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of Interest"}},{"value":"Not applicable","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Clinical trial number"}}],"article-number":"6"}}