{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T18:00:32Z","timestamp":1781632832502,"version":"3.54.5"},"reference-count":105,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2026,2,7]],"date-time":"2026-02-07T00:00:00Z","timestamp":1770422400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,2,7]],"date-time":"2026-02-07T00:00:00Z","timestamp":1770422400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000995","name":"Australian National University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100000995","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2026,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Expedient design choices in software development can lead to Technical Debt (TD), with development teams documenting such decisions as Self-Admitted TD (SATD). Algorithm Debt (AD) is a type of TD resulting from the suboptimal implementation of algorithms, which impacts system performance. Given the impact of AD, its automated detection is crucial in Deep Learning (DL) frameworks due to their complexity and evolution. Early detection of AD in DL frameworks can help mitigate model degradation and scalability issues. Despite previous studies on the automated detection of TD from SATD using Machine Learning (ML)\/DL models, research on AD detection in DL frameworks remains underexplored. In this study, we empirically investigated the performance of ML\/DL models for the automated detection of AD using a dataset of 38,\u00a0881 SATD comments from seven DL frameworks. We trained, evaluated, and tested ML\/DL models, used embeddings from both DL and large language models, and explored an approach to enrich the dataset with handcrafted features based on AD-related keywords. Our findings reveal that AD is frequently misclassified as Design or Implementation Debt. Logistic Regression (an ML model) with Custom AD Features, achieved an F1-score of 54% for AD, outperforming other ML\/DL models (42% to 52%), highlighting the importance of tailored feature engineering. Our research advances automated AD detection in DL frameworks by providing insights into the strengths and limitations of ML\/DL models, serving as a first step to guide future tool development. This could help developers using DL frameworks to identify AD issues during development, thereby enhancing system reliability by mitigating model degradation and scalability challenges.<\/jats:p>","DOI":"10.1007\/s10664-026-10807-5","type":"journal-article","created":{"date-parts":[[2026,2,7]],"date-time":"2026-02-07T10:27:08Z","timestamp":1770460028000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Automated detection of algorithm debt in deep learning frameworks: an empirical study"],"prefix":"10.1007","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8494-6885","authenticated-orcid":false,"given":"Emmanuel Iko-Ojo","family":"Simon","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chirath","family":"Hettiarachchi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alex","family":"Potanin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hanna","family":"Suominen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fatemeh","family":"Fard","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,2,7]]},"reference":[{"key":"10807_CR1","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2021.102693","volume":"213","author":"EA AlOmar","year":"2022","unstructured":"AlOmar EA, Christians B, Busho M, AlKhalid AH, Ouni A, Newman C (2022) Satdbailiff-mining and tracking self-admitted technical debt. Sci Comput Program 213:102693","journal-title":"Sci Comput Program"},{"key":"10807_CR2","first-page":"1","volume-title":"Int","author":"N Alves","year":"2014","unstructured":"Alves N, Ribeiro LF, Caires V, Mendes TS, Sp\u00ednola RO (2014) Towards an ontology of terms on technical debt. Int. Workshop on Managing Technical Debt, IEEE, pp 1\u20137"},{"key":"10807_CR3","unstructured":"Ameisen E (2021) Building Machine Learning Powered Applications: Going from Idea to Product. O\u2019Reilly Media"},{"key":"10807_CR4","doi-asserted-by":"crossref","unstructured":"Arisholm E, Briand LC, Fuglerud M (2007) Data mining techniques for building fault-proneness models in telecom java software. In: 18th IEEE International Symposium on Software Reliability, IEEE, pp 215\u2013224","DOI":"10.1109\/ISSRE.2007.22"},{"key":"10807_CR5","doi-asserted-by":"crossref","unstructured":"Arsyah UI, Pratiwi M, Muhammad A (2024) Twitter sentiment analysis of public space opinions using svm and tf-idf methods. Indonesian Journal of Computer Science 13(1)","DOI":"10.33022\/ijcs.v13i1.3594"},{"key":"10807_CR6","unstructured":"Avgeriou P, Kruchten P, Ozkaya I, Seaman C (2016) Managing technical debt in software engineering (dagstuhl seminar 16162). In: Dagstuhl Reports, Schloss Dagstuhl-Leibniz-Zentrum f\u00fcr Informatik, vol\u00a06"},{"key":"10807_CR7","doi-asserted-by":"crossref","unstructured":"Ayyagari M, Bhatt VD, Pandey A (2022) Efficient implementation of pooling operation for ai accelerators. In: 2022 IEEE International Conference for Women in Innovation, Technology & Entrepreneurship (ICWITE), IEEE, pp 1\u20135","DOI":"10.1109\/ICWITE57052.2022.10176216"},{"issue":"2","key":"10807_CR8","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1007\/s10664-021-10081-7","volume":"27","author":"H Azuma","year":"2022","unstructured":"Azuma H, Matsumoto S, Kamei Y, Kusumoto S (2022) An empirical study on self-admitted technical debt in dockerfiles. Empir Softw Eng 27(2):49","journal-title":"Empir Softw Eng"},{"issue":"1","key":"10807_CR9","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1145\/1007730.1007735","volume":"6","author":"GE Batista","year":"2004","unstructured":"Batista GE, Prati RC, Monard MC (2004) A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explorations Newsl 6(1):20\u201329","journal-title":"ACM SIGKDD Explorations Newsl"},{"key":"10807_CR10","doi-asserted-by":"crossref","unstructured":"Bavota G, Russo B (2016) A large-scale empirical study on self-admitted technical debt. In: Proceedings of the 13th international conference on mining software repositories, pp 315\u2013326","DOI":"10.1145\/2901739.2901742"},{"key":"10807_CR11","unstructured":"Bergstra J, Bardenet R, Bengio Y, K\u00e9gl B (2011) Algorithms for hyper-parameter optimization. Advances in Neural Information Processing Systems 24"},{"key":"10807_CR12","doi-asserted-by":"crossref","unstructured":"Bertsimas D, King A (2017) Logistic regression: From art to science. Statistical Science pp 367\u2013384","DOI":"10.1214\/16-STS602"},{"key":"10807_CR13","doi-asserted-by":"crossref","unstructured":"Bhatia A, Khomh F, Adams B, Hassan AE (2024) An empirical study of self-admitted technical debt in machine learning software. ACM Transaction on Software Engineering Methodologies 1(1)","DOI":"10.1145\/3785001"},{"key":"10807_CR14","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1007\/s11749-016-0481-7","volume":"25","author":"G Biau","year":"2016","unstructured":"Biau G, Scornet E (2016) A random forest guided tour. TEST 25:197\u2013227","journal-title":"TEST"},{"key":"10807_CR15","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L (2001) Random forests Machine learning 45:5\u201332","journal-title":"Random forests Machine learning"},{"issue":"1","key":"10807_CR16","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1177\/001316446002000104","volume":"20","author":"J Cohen","year":"1960","unstructured":"Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Measur 20(1):37\u201346","journal-title":"Educ Psychol Measur"},{"key":"10807_CR17","first-page":"273","volume":"20","author":"C Cortes","year":"1995","unstructured":"Cortes C, Vapnik V (1995) Support-vector networks Machine learning 20:273\u2013297","journal-title":"Support-vector networks Machine learning"},{"key":"10807_CR18","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1145\/157710.157715","volume":"4","author":"W Cunningham","year":"1992","unstructured":"Cunningham W (1992) The wycash portfolio mgt system. ACM SIGPLAN OOPS 4:29\u201330","journal-title":"ACM SIGPLAN OOPS"},{"issue":"11","key":"10807_CR19","doi-asserted-by":"publisher","first-page":"1044","DOI":"10.1109\/TSE.2017.2654244","volume":"43","author":"ME da Silva","year":"2017","unstructured":"da Silva ME, Shihab E, Tsantalis N (2017) Using natural language processing to automatically detect satd. IEEE Trans Software Eng 43(11):1044\u20131062","journal-title":"IEEE Trans Software Eng"},{"issue":"1","key":"10807_CR20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s13278-024-01250-9","volume":"14","author":"MM Danyal","year":"2024","unstructured":"Danyal MM, Khan SS, Khan M, Ullah S, Ghaffar MB, Khan W (2024) Sentiment analysis of movie reviews based on nb approaches using tf-idf and count vectorizer. Soc Netw Anal Min 14(1):1\u201315","journal-title":"Soc Netw Anal Min"},{"key":"10807_CR21","first-page":"786","volume-title":"2019 Third International conference on I-SMAC (IoT in Social","author":"D Deepa","year":"2019","unstructured":"Deepa D, Tamilarasi A et al (2019) Sentiment analysis using feature extraction and dictionary-based approaches. 2019 Third International conference on I-SMAC (IoT in Social. Mobile, Analytics and Cloud)(I-SMAC), IEEE, pp 786\u2013790"},{"key":"10807_CR22","doi-asserted-by":"publisher","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Burstein J, Doran C, Solorio T (eds) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, Minneapolis, Minnesota, pp 4171\u20134186, https:\/\/doi.org\/10.18653\/v1\/N19-1423, https:\/\/aclanthology.org\/N19-1423\/","DOI":"10.18653\/v1\/N19-1423"},{"key":"10807_CR23","doi-asserted-by":"crossref","unstructured":"Ebrahimi AM, Oliva GA, Hassan AE (2023) Self-admitted technical debt in ethereum smart contracts: A large-scale exploratory study. IEEE Trans Softw Eng","DOI":"10.1109\/TSE.2023.3289808"},{"key":"10807_CR24","doi-asserted-by":"crossref","unstructured":"Feuerriegel S, Maarouf A, B\u00e4r D, Geissler D, Schweisthal J, Pr\u00f6llochs N, Robertson CE, Rathje S, Hartmann J, Mohammad SM, et\u00a0al (2025) Using natural language processing to analyse text data in behavioural science. Nat Rev Psychol 1\u201316","DOI":"10.1038\/s44159-024-00392-z"},{"issue":"10","key":"10807_CR25","doi-asserted-by":"publisher","first-page":"e0291908","DOI":"10.1371\/journal.pone.0291908","volume":"18","author":"GM Foody","year":"2023","unstructured":"Foody GM (2023) Challenges in the real world use of classification accuracy metrics: From recall and precision to the matthews correlation coefficient. PLoS One 18(10):e0291908","journal-title":"PLoS One"},{"key":"10807_CR26","doi-asserted-by":"crossref","unstructured":"Ghasemi N, Justo JA, Celesti M, Despoisse L, Nieke J (2025) Onboard processing of hyperspectral imagery: Deep learning advancements, methodologies, challenges, and emerging trends. IEEE J Sel Top Appl Earth Observ Remote Sens","DOI":"10.1109\/JSTARS.2025.3527898"},{"key":"10807_CR27","unstructured":"Goodfellow I, Bengio Y, Courville A (2016) Deep Learning, 1st edn. MIT Press, Cambridge, MA. https:\/\/www.deeplearningbook.org\/"},{"key":"10807_CR28","volume-title":"Feature extraction: foundations and applications,","author":"I Guyon","year":"2008","unstructured":"Guyon I, Gunn S, Nikravesh M, Zadeh LA (2008) Feature extraction: foundations and applications, vol 207. Springer"},{"key":"10807_CR29","doi-asserted-by":"crossref","unstructured":"Hilbe JM (2009) Logistic regression models. Chapman and hall\/CRC","DOI":"10.1201\/9781420075779"},{"key":"10807_CR30","doi-asserted-by":"publisher","first-page":"418","DOI":"10.1007\/s10664-017-9522-4","volume":"23","author":"Q Huang","year":"2018","unstructured":"Huang Q, Shihab E, Xia X, Lo D, Li S (2018) Identifying self-admitted technical debt in open source projects using text mining. Empir Softw Eng 23:418\u2013451","journal-title":"Empir Softw Eng"},{"key":"10807_CR31","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-7138-7","volume-title":"An introduction to statistical learning,","author":"G James","year":"2013","unstructured":"James G, Witten D, Hastie T, Tibshirani R et al (2013) An introduction to statistical learning, vol 112. Springer"},{"key":"10807_CR32","doi-asserted-by":"crossref","unstructured":"Joachims T (1998) Text categorization with support vector machines: Learning with many relevant features. In: European Conference on Machine Learning, Springer, pp 137\u2013142","DOI":"10.1007\/BFb0026683"},{"key":"10807_CR33","doi-asserted-by":"crossref","unstructured":"Joulin A, Grave E, Bojanowski P, Mikolov T (2017) Bag of tricks for efficient text classification. In: Lapata M, Blunsom P, Koller A (eds) Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, Association for Computational Linguistics, Valencia, Spain, pp 427\u2013431. https:\/\/aclanthology.org\/E17-2068\/","DOI":"10.18653\/v1\/E17-2068"},{"issue":"4","key":"10807_CR34","first-page":"207","volume":"4","author":"J Kaur","year":"2018","unstructured":"Kaur J, Buttar PK (2018) A systematic review on stopword removal algorithms. Int J Future Revol Comput Sci Commun Eng 4(4):207\u2013210","journal-title":"Int J Future Revol Comput Sci Commun Eng"},{"issue":"1","key":"10807_CR35","first-page":"155","volume":"3","author":"AK Khan","year":"2024","unstructured":"Khan AK (2024) Ai in finance disruptive technologies and emerging opportunities. J Artif Intell Gen Sci (JAIGS) 3(1):155\u2013170","journal-title":"J Artif Intell Gen Sci (JAIGS)"},{"key":"10807_CR36","doi-asserted-by":"crossref","unstructured":"Khoshgoftaar TM, Golawala M, Van\u00a0Hulse J (2007) An empirical study of learning from imbalanced data using random forest. In: 19th IEEE international conference on tools with artificial intelligence (ICTAI 2007), IEEE, vol\u00a02, pp 310\u2013317","DOI":"10.1109\/ICTAI.2007.46"},{"key":"10807_CR37","unstructured":"Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2019) Albert: A lite bert for self-supervised learning of language representations. In: 2020 International conference on learning representations, ICLR \u201920"},{"key":"10807_CR38","doi-asserted-by":"crossref","unstructured":"Lemieux C, Inala JP, Lahiri, Sen S (2023) Codamosa: Escaping coverage plateaus in test generation with pre-trained large language models. In: 2023 IEEE\/ACM 45th international conference on software engineering (ICSE), IEEE, pp 919\u2013931","DOI":"10.1109\/ICSE48619.2023.00085"},{"key":"10807_CR39","doi-asserted-by":"crossref","unstructured":"Lenarduzzi V, Saarim\u00e4ki N, Taibi D (2019) The technical debt dataset. In: Proceedings of the fifteenth international conference on predictive models and data analytics in software engineering, pp 2\u201311","DOI":"10.1145\/3345629.3345630"},{"issue":"3","key":"10807_CR40","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10664-023-10297-9","volume":"28","author":"Y Li","year":"2023","unstructured":"Li Y, Soliman M, Avgeriou P (2023) Automatic identification of self-admitted technical debt from four different sources. Empir Softw Eng 28(3):1\u201338","journal-title":"Empir Softw Eng"},{"key":"10807_CR41","doi-asserted-by":"crossref","unstructured":"Li J, Li L, Liu J, Yu X, Liu X, Keung JW (2025) Large language model chatgpt versus small deep learning models for self-admitted technical debt detection: Why not together? Softw Pract Exper 55(1):3\u201328","DOI":"10.1002\/spe.3360"},{"key":"10807_CR42","doi-asserted-by":"publisher","unstructured":"Li J, Mani G (2021) Machine learning application on prediction of male breast cancer with plco dataset. J Student Res 10(3). https:\/\/doi.org\/10.47611\/jsrhs.v10i3.2199","DOI":"10.47611\/jsrhs.v10i3.2199"},{"key":"10807_CR43","doi-asserted-by":"publisher","unstructured":"Li Y, Soliman M, Avgeriou P (2023b) Automatically identifying relations between self-admitted technical debt across different sources. In: 2023 ACM\/IEEE international conference on technical debt (TechDebt), pp 11\u201321. https:\/\/doi.org\/10.1109\/TechDebt59074.2023.00008","DOI":"10.1109\/TechDebt59074.2023.00008"},{"key":"10807_CR44","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10664-020-09917-5","volume":"26","author":"J Liu","year":"2021","unstructured":"Liu J, Huang Q, Xia X, Shihab E, Lo D, Li S (2021) An exploratory study on the introduction and removal of different types of technical debt in deep learning frameworks. Empir Softw Eng 26:1\u201336","journal-title":"Empir Softw Eng"},{"issue":"3","key":"10807_CR45","doi-asserted-by":"publisher","first-page":"1046","DOI":"10.3390\/app14031046","volume":"14","author":"M Liu","year":"2024","unstructured":"Liu M, Wang J, Lin T, Ma Q, Fang Z, Wu Y (2024) An empirical study of the code generation of safety-critical software using llms. Appl Sci 14(3):1046","journal-title":"Appl Sci"},{"key":"10807_CR46","doi-asserted-by":"crossref","unstructured":"Liu J, Huang Q, Xia X, Shihab E, Lo D, Li S (2020) Is using deep learning frameworks free? characterizing technical debt in deep learning frameworks. In: ACM\/IEEE 42nd int conf on software engineering, pp 1\u201310","DOI":"10.1145\/3377815.3381377"},{"key":"10807_CR47","doi-asserted-by":"publisher","unstructured":"Liu Z, Lin W, Shi Y, Zhao J (2021b) A robustly optimized bert pre-training approach with post-training. In: Chinese Computational Linguistics: 20th China National Conference, CCL 2021, Hohhot, China, August 13\u201315, 2021, Proceedings, Springer-Verlag, Berlin, Heidelberg, pp 471\u2013484. https:\/\/doi.org\/10.1007\/978-3-030-84186-7_31https:\/\/doi.org\/10.1007\/978-3-030-84186-7_31","DOI":"10.1007\/978-3-030-84186-7_31"},{"key":"10807_CR48","unstructured":"Loureiro D (2023) Learning word sense representations from neural language models. PhD thesis, Universidade do Porto (Portugal)"},{"issue":"2","key":"10807_CR49","doi-asserted-by":"publisher","first-page":"545","DOI":"10.11144\/Javeriana.upsy10-2.cdcp","volume":"10","author":"G Macbeth","year":"2011","unstructured":"Macbeth G, Razumiejczyk E, Ledesma RD (2011) Cliff\u2019s delta calculator: A non-parametric effect size program for two groups of observations. Universitas Psychologica 10(2):545\u2013555","journal-title":"Universitas Psychologica"},{"key":"10807_CR50","doi-asserted-by":"crossref","unstructured":"Maldonado EdS, Abdalkareem R, Shihab E, Serebrenik A (2017) An empirical study on the removal of self-admitted technical debt. In: International Conference on Software Maintenance and Evolution (ICSME), IEEE, pp 238\u2013248","DOI":"10.1109\/ICSME.2017.8"},{"key":"10807_CR51","doi-asserted-by":"crossref","unstructured":"Maldonado EdS, Shihab E (2015) Detecting and quantifying different types of self-admitted technical debt. In: 7th International Workshop on Managing Technical Debt, IEEE, pp 9\u201315","DOI":"10.1109\/MTD.2015.7332619"},{"key":"10807_CR52","doi-asserted-by":"crossref","unstructured":"Mastropaolo A, Di\u00a0Penta M, Bavota G (2023) Towards automatically addressing self-admitted technical debt: How far are we? In: 2023 38th IEEE\/ACM international conference on automated software engineering (ASE), IEEE, pp 585\u2013597","DOI":"10.1109\/ASE56229.2023.00103"},{"key":"10807_CR53","unstructured":"Meissel K, Yao ES (2024) Using cliff\u2019s delta as a non-parametric effect size measure: an accessible web app and r tutorial. Practical Assessment, Research, and Evaluation 29(1)"},{"issue":"559\u2013569","key":"10807_CR54","first-page":"4","volume":"10","author":"WID Mining","year":"2006","unstructured":"Mining WID (2006) Data mining: Concepts and techniques. Morgan Kaufinann 10(559\u2013569):4","journal-title":"Morgan Kaufinann"},{"key":"10807_CR55","doi-asserted-by":"publisher","unstructured":"Moreschini S, Lenarduzzi V, Coba L (2024) Towards a technical debt for ai-based recommender system. In: Proceedings of the 7th ACM\/IEEE International Conference on Technical Debt, Association for Computing Machinery, New York, NY, USA, TechDebt \u201924, pp 36\u201339. https:\/\/doi.org\/10.1145\/3644384.3648574","DOI":"10.1145\/3644384.3648574"},{"key":"10807_CR56","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1007\/s13042-012-0068-x","volume":"4","author":"AB Musa","year":"2013","unstructured":"Musa AB (2013) Comparative study on classification performance between support vector machine and logistic regression. Int J Mach Learn Cybern 4:13\u201324","journal-title":"Int J Mach Learn Cybern"},{"key":"10807_CR57","doi-asserted-by":"crossref","unstructured":"Nunes RO, Spritzer AS, Balreira DG, Freitas CM, Carbonera JL (2024) An evaluation of large language models for geological named entity recognition. IEEE","DOI":"10.1109\/ICTAI62512.2024.00076"},{"key":"10807_CR58","doi-asserted-by":"crossref","unstructured":"OBrien D, Biswas S, Imtiaz S, Abdalkareem R, Shihab E, Rajan H (2022) 23 shades of self-admitted technical debt: an empirical study on machine learning software. In: Proceedings of the 30th ACM joint european software engineering conference and symposium on the foundations of software engineering, pp 734\u2013746","DOI":"10.1145\/3540250.3549088"},{"issue":"3","key":"10807_CR59","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1109\/MS.2023.3248401","volume":"40","author":"I Ozkaya","year":"2023","unstructured":"Ozkaya I (2023) Application of llms to software engineering tasks: Opportunities, risks, and implications. IEEE Softw 40(3):4\u20138","journal-title":"IEEE Softw"},{"key":"10807_CR60","doi-asserted-by":"crossref","unstructured":"Pepe F, Zampetti F, Mastropaolo A, Bavota G, Di\u00a0Penta M (2024) A taxonomy of self-admitted technical debt in deep learning systems. In: 2024 IEEE international conference on software maintenance and evolution (ICSME), pp 388\u2013399","DOI":"10.1109\/ICSME58944.2024.00043"},{"issue":"1","key":"10807_CR61","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1007\/s10664-021-10045-x","volume":"27","author":"A Peruma","year":"2022","unstructured":"Peruma A, Simmons S, AlOmar EA, Newman CD, Mkaouer MW, Ouni A (2022) How do i refactor this? an empirical study on refactoring trends and topics in stack overflow. Empir Softw Eng 27(1):11. https:\/\/doi.org\/10.1007\/s10664-021-10045-x","journal-title":"Empir Softw Eng"},{"issue":"7","key":"10807_CR62","doi-asserted-by":"publisher","first-page":"232","DOI":"10.3390\/fi15070232","volume":"15","author":"A Pinna","year":"2023","unstructured":"Pinna A, Lunesu MI, Orr\u00f9 S, Tonelli R (2023) Investigation on self-admitted technical debt in open-source blockchain projects. Future Int 15(7):232","journal-title":"Future Int"},{"key":"10807_CR63","doi-asserted-by":"crossref","unstructured":"Poczeta K, P\u0142aza M, Michno T, Krechowicz M, Zawadzki M (2023) A multi-label text message classification method designed for applications in call\/contact centre systems. Appl Soft Comput 145","DOI":"10.1016\/j.asoc.2023.110562"},{"key":"10807_CR64","doi-asserted-by":"crossref","unstructured":"Potdar A, Shihab E (2014) An exploratory study on self-admitted technical debt. In: Int conference on software maintenance and evolution, pp 91\u2013100","DOI":"10.1109\/ICSME.2014.31"},{"key":"10807_CR65","doi-asserted-by":"publisher","first-page":"154704","DOI":"10.1109\/ACCESS.2021.3128742","volume":"9","author":"P Rajapaksha","year":"2021","unstructured":"Rajapaksha P, Farahbakhsh R, Crespi N (2021) Bert, xlnet or roberta: the best transfer learning model to detect clickbaits. IEEE Access 9:154704\u2013154716","journal-title":"IEEE Access"},{"key":"10807_CR66","unstructured":"Rantala L, Lenarduzzi V, M\u00e4ntyl\u00e4 MV (2024) Relationship between self-admitted technical debt and code-level technical debt. an empirical evaluation. Softw Quality Jornal. https:\/\/www.researchgate.net\/publication\/374998443_Relationship_between_Self-Admitted_Technical_Debt_and_Code-level_Technical_Debt_An_Empirical_Evaluation"},{"key":"10807_CR67","doi-asserted-by":"crossref","unstructured":"Recupito G, Pecorelli F, Catolino G, Lenarduzzi V, Taibi D, Di\u00a0Nucci D, Palomba F (2024) Technical debt in ai-enabled systems: On the prevalence, severity, impact, and management strategies for code and architecture. J Syst Softw 112151","DOI":"10.1016\/j.jss.2024.112151"},{"issue":"3","key":"10807_CR68","first-page":"127","volume":"1","author":"Y Rejani","year":"2009","unstructured":"Rejani Y, Selvi ST (2009) Early detection of breast cancer using svm classifier technique. Int J Comput Eng 1(3):127\u2013130","journal-title":"Int J Comput Eng"},{"issue":"3","key":"10807_CR69","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3324916","volume":"28","author":"X Ren","year":"2019","unstructured":"Ren X, Xing Z, Xia X, Lo D, Wang X, Grundy J (2019) Neural network-based detection of self-admitted technical debt: From performance to explainability. ACM Trans Softw Eng Method (TOSEM) 28(3):1\u201345","journal-title":"ACM Trans Softw Eng Method (TOSEM)"},{"key":"10807_CR70","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1016\/j.infsof.2018.05.010","volume":"102","author":"N Rios","year":"2018","unstructured":"Rios N, de MendonCa Neto MG, SpInola RO (2018) A tertiary study on technical debt: Types, management strategies, research trends, and base information for practitioners. Inf Softw Technol 102:117\u2013145","journal-title":"Inf Softw Technol"},{"issue":"1","key":"10807_CR71","first-page":"5","volume":"46","author":"R Roshan","year":"2023","unstructured":"Roshan R, Bhacho IA, Zai S (2023) Comparative analysis of tf-idf and hashing vectoriser for fake news detection in sindhi: A machine learning and deep learning approach. Eng Proc 46(1):5","journal-title":"Eng Proc"},{"key":"10807_CR72","doi-asserted-by":"crossref","unstructured":"Sabbah AF, Hanani AA (2023) Self-admitted technical debt classification using natural language processing word embeddings. Int J Electr Comp Engr 13(2)","DOI":"10.11591\/ijece.v13i2.pp2142-2155"},{"key":"10807_CR73","doi-asserted-by":"crossref","unstructured":"Sallou J, Durieux T, Panichella A (2024) Breaking the silence: the threats of using llms in software engineering. In: Proceedings of the 2024 ACM\/IEEE 44th International conference on software engineering: new ideas and emerging results, pp 102\u2013106","DOI":"10.1145\/3639476.3639764"},{"key":"10807_CR74","doi-asserted-by":"crossref","unstructured":"Santos RM, Santos IM, J\u00fanior MCR, de\u00a0Mendon\u00e7a\u00a0Neto MG (2020) Long term-short memory neural networks and word2vec for self-admitted technical debt detection. In: International conference on enterprise information systems (ICEIS), pp 157\u2013165","DOI":"10.5220\/0009796001570165"},{"issue":"3","key":"10807_CR75","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1007\/s42979-021-00592-x","volume":"2","author":"IH Sarker","year":"2021","unstructured":"Sarker IH (2021) Machine learning: Algorithms, real-world applications and research directions. SN Comput Sci 2(3):160","journal-title":"SN Comput Sci"},{"key":"10807_CR76","unstructured":"Sculley D, Holt G, Golovin D, Davydov E, Phillips T, Ebner D, Chaudhary V, Young JF, Dennison D (2015) Hidden technical debt in machine learning systems. Adv Neural Inf Process Syst 28"},{"issue":"1","key":"10807_CR77","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1007\/s41133-020-00032-0","volume":"5","author":"K Shah","year":"2020","unstructured":"Shah K, Patel H, Sanghvi D, Shah M (2020) A comparative analysis of logistic regression, random forest and knn models for the text classification. Augmented Human Res 5(1):12","journal-title":"Augmented Human Res"},{"issue":"2","key":"10807_CR78","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1007\/s10515-022-00358-6","volume":"29","author":"R Sharma","year":"2022","unstructured":"Sharma R, Shahbazi R, Fard FH, Codabux Z, Vidoni M (2022) Self-admitted technical debt in r: detection and causes. Autom Softw Eng 29(2):53","journal-title":"Autom Softw Eng"},{"issue":"6","key":"10807_CR79","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1007\/s10664-024-10548-3","volume":"29","author":"MS Sheikhaei","year":"2024","unstructured":"Sheikhaei MS, Tian Y, Wang S, Xu B (2024) An empirical study on the effectiveness of large language models for satd identification and classification. Empir Softw Eng 29(6):159","journal-title":"Empir Softw Eng"},{"key":"10807_CR80","doi-asserted-by":"publisher","unstructured":"Simon EIO, Hettiarachchi C, Potanin A, Suominen H, Fard F (2024) Automated detection of Algorithm Debt in deep learning frameworks: An empirical study. Presented at the 40th IEEE international conference on software maintenance and evolution (ICSME), Registered Reports Track. https:\/\/doi.org\/10.48550\/arXiv.2408.10529","DOI":"10.48550\/arXiv.2408.10529"},{"key":"10807_CR81","doi-asserted-by":"crossref","unstructured":"Simon EIO, Vidoni M, Fard FH (2023) Algorithm debt: Challenges and future paths. In: 2023 IEEE\/ACM 2nd international conference on AI engineering\u2013software engineering for AI, IEEE, pp 90\u201391","DOI":"10.1109\/CAIN58948.2023.00020"},{"key":"10807_CR82","doi-asserted-by":"crossref","unstructured":"Sklavenitis D, Kalles D (2024) Measuring technical debt in ai-based competition platforms. In: Proceedings of the 13th hellenic conference on artificial intelligence, pp 1\u201310","DOI":"10.1145\/3688671.3688783"},{"key":"10807_CR83","unstructured":"Suominen H, Pahikkala T, Salakoski T (2008) Critical points in assessing learning performance via cross-validation. In: Proceedings of the 2nd international and interdisciplinary conference on adaptive knowledge representation and reasoning (AKRR 2008), Helsinki University of Technology, pp 9\u201322"},{"key":"10807_CR84","doi-asserted-by":"crossref","unstructured":"Suryaningrum KM (2023) Comparison of the tf-idf method with the count vectorizer to classify hate speech. Eng Math Comput Sci J (EMACS) 5(2):79\u201383","DOI":"10.21512\/emacsjournal.v5i2.9978"},{"key":"10807_CR85","doi-asserted-by":"crossref","unstructured":"Su H, Shi W, Kasai J, Wang Y, Hu Y, Ostendorf M, Yih Wt, Smith NA, Zettlemoyer L, Yu T (2023) One embedder, any task: Instruction-finetuned text embeddings. In: Findings of the association for computational linguistics: ACL 2023","DOI":"10.18653\/v1\/2023.findings-acl.71"},{"key":"10807_CR86","doi-asserted-by":"crossref","unstructured":"Sutoyo E, Avgeriou P, Capiluppi A (2024) Deep learning and data augmentation for detecting self-admitted technical debt. In: 31st Asia-pacific software engineering conference (APSEC 2024)","DOI":"10.1109\/APSEC65559.2024.00022"},{"key":"10807_CR87","doi-asserted-by":"publisher","unstructured":"Sutoyo E, Capiluppi A (2024) Satdaug - a balanced and augmented dataset for detecting self-admitted technical debt. In: Proceedings of the 21st international conference on mining software repositories, association for computing machinery, New York, NY, USA, MSR \u201924, pp 289\u2013293. https:\/\/doi.org\/10.1145\/3643991.3644880","DOI":"10.1145\/3643991.3644880"},{"key":"10807_CR88","doi-asserted-by":"crossref","unstructured":"Tang Y, Khatchadourian R, Bagherzadeh M, Singh R, Stewart A, Raja A (2021) An empirical study of refactorings and technical debt in machine learning systems. In: 2021 IEEE\/ACM 43rd international conference on software engineering (ICSE), IEEE, pp 238\u2013250","DOI":"10.1109\/ICSE43902.2021.00033"},{"key":"10807_CR89","unstructured":"Tang Y, Yang Y (2024) Multihop-RAG: Benchmarking retrieval-augmented generation for multi-hop queries. In: First conference on language modeling. https:\/\/openreview.net\/forum?id=t4eB3zYWBK"},{"issue":"2","key":"10807_CR90","first-page":"101","volume":"25","author":"A Vargha","year":"2000","unstructured":"Vargha A, Delaney HD (2000) A critique and improvement of the cl common language effect size statistics of mcgraw and wong. J Educ Behav Stat 25(2):101\u2013132","journal-title":"J Educ Behav Stat"},{"key":"10807_CR91","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Lu, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, curran associates, Inc., vol\u00a030. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf"},{"key":"10807_CR92","doi-asserted-by":"crossref","unstructured":"Venkatkrishna V, Nagabushanam DS, Simon EIO, Vidoni M (2024) Multi-step automated generation of parameter docstrings in python: An exploratory study. In: Proceedings of the 2024 IEEE\/ACM 46th international conference on software engineering: companion proceedings, pp 356\u2013357","DOI":"10.1145\/3639478.3643110"},{"key":"10807_CR93","doi-asserted-by":"crossref","unstructured":"Vidoni M (2021) Self-admitted technical debt in r packages: An exploratory study. In: International Conference on Mining Software Repositories, IEEE, pp 179\u2013189","DOI":"10.1109\/MSR52588.2021.00030"},{"key":"10807_CR94","doi-asserted-by":"publisher","unstructured":"Wang Q, Li B, Xiao T, Zhu J, Li C, Wong DF, Chao LS (2019) Learning deep transformer models for machine translation. In: Proceedings of the 57th annual meeting of the association for computational linguistics, association for computational linguistics, Florence, Italy, pp 1810\u20131822. https:\/\/doi.org\/10.18653\/v1\/P19-1176https:\/\/aclanthology.org\/P19-1176\/","DOI":"10.18653\/v1\/P19-1176"},{"key":"10807_CR95","doi-asserted-by":"crossref","unstructured":"Wattanakriengkrai S, Srisermphoak N, Sintoplertchaikul S, Choetkiertikul M, Ragkhitwetsagul C, Sunetnanta T, Hata H, Matsumoto K (2019) Automatic classifying self-admitted technical debt using n-gram idf. In: 2019 26th Asia-Pacific software engineering conference (APSEC), IEEE, pp 316\u2013322","DOI":"10.1109\/APSEC48747.2019.00050"},{"key":"10807_CR96","first-page":"171","volume":"1","author":"F Wilcoxon","year":"1970","unstructured":"Wilcoxon F, Katti S, Wilcox RA et al (1970) Critical values and probability levels for the wilcoxon rank sum test and the wilcoxon signed rank test. Sel Tables Math Stat 1:171\u2013259","journal-title":"Sel Tables Math Stat"},{"issue":"1","key":"10807_CR97","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1007\/s40593-023-00342-5","volume":"34","author":"B Williamson","year":"2024","unstructured":"Williamson B (2024) The social life of ai in education. Int J Artif Intell Educ 34(1):97\u2013104","journal-title":"Int J Artif Intell Educ"},{"issue":"2","key":"10807_CR98","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10664-024-10449-5","volume":"29","author":"T Xiao","year":"2024","unstructured":"Xiao T, Zeng Z, Wang D, McIntosh S (2024) Quantifying and characterizing clones of satd in build systems. Empir Softw Eng 29(2):1\u201331","journal-title":"Empir Softw Eng"},{"key":"10807_CR99","unstructured":"Ximenes RG (2024) Issues that lead to code technical debt in machine learning systems. Master\u2019s dissertation, Pontifical Catholic University of Rio de Janeiro (PUC-Rio)"},{"key":"10807_CR100","doi-asserted-by":"crossref","unstructured":"Yadav S, Shukla S (2016) Analysis of k-fold cross-validation over hold-out validation on colossal datasets for quality classification. In: 2016 IEEE 6th international conference on advanced computing (IACC), IEEE, pp 78\u201383","DOI":"10.1109\/IACC.2016.25"},{"key":"10807_CR101","doi-asserted-by":"crossref","unstructured":"Yang Y, Li J, Yang Y (2015) The research of the fast svm classifier method. In: 2015 12th International computer conference on wavelet active media technology and information processing (ICCWAMTIP), IEEE, pp 121\u2013124","DOI":"10.1109\/ICCWAMTIP.2015.7493959"},{"issue":"5","key":"10807_CR102","doi-asserted-by":"publisher","first-page":"1676","DOI":"10.1109\/TSE.2020.3031401","volume":"48","author":"Z Yu","year":"2020","unstructured":"Yu Z, Fahid FM, Tu H, Menzies T (2020) Identifying self-admitted technical debts with jitterbug: A two-step approach. IEEE Trans Software Eng 48(5):1676\u20131691","journal-title":"IEEE Trans Software Eng"},{"issue":"23","key":"10807_CR103","doi-asserted-by":"publisher","first-page":"4700","DOI":"10.3390\/electronics13234700","volume":"13","author":"J Yu","year":"2024","unstructured":"Yu J, Tian H, Li R, Zuo Q, Di Y (2024) Detecting self-admitted technical debts via prompt-based method in issue-tracking systems. Electronics 13(23):4700. https:\/\/doi.org\/10.3390\/electronics13234700","journal-title":"Electronics"},{"key":"10807_CR104","doi-asserted-by":"crossref","unstructured":"Yu J, Zhou X, Liu X, Liu J, Xie Z, Zhao K (2023) Detecting multi-type satd with gan-based neural networks. Inf Softw Technol 158","DOI":"10.1016\/j.infsof.2023.107190"},{"key":"10807_CR105","first-page":"355","volume-title":"2020 IEEE 27th International Conference on Software Analysis","author":"F Zampetti","year":"2020","unstructured":"Zampetti F, Serebrenik A, Di Penta M (2020) Automatically learning patterns for self-admitted technical debt removal. 2020 IEEE 27th International Conference on Software Analysis. Evolution and Reengineering (SANER), IEEE, pp 355\u2013366"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-026-10807-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-026-10807-5","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-026-10807-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T17:44:55Z","timestamp":1781631895000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-026-10807-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,7]]},"references-count":105,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,5]]}},"alternative-id":["10807"],"URL":"https:\/\/doi.org\/10.1007\/s10664-026-10807-5","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,7]]},"assertion":[{"value":"26 May 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 January 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 February 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no Conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed Consent"}},{"value":"Clinical trial number: Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Clinical Trial Number"}}],"article-number":"66"}}