{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,10]],"date-time":"2026-11-10T10:45:11Z","timestamp":1794307511909,"version":"4.9.0"},"reference-count":101,"publisher":"Association for Computing Machinery (ACM)","issue":"8","license":[{"start":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T00:00:00Z","timestamp":1784246400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,8,31]]},"abstract":"<jats:p>\n                    The emergence of open source ML libraries such as TensorFlow and Google Auto ML has enabled developers to harness state-of-the-art ML algorithms with minimal overhead. However, during this accelerated ML development process, said developers may often make sub-optimal design and implementation decisions, leading to the introduction of technical debt that, if not addressed promptly, can significantly impact on the quality of ML-based software. Developers frequently acknowledge these sub-optimal design and development choices through code comments written during development. These comments, which often highlight areas requiring additional work or refinement in the future are known as\n                    <jats:italic toggle=\"yes\">self-admitted technical debt (SATD)<\/jats:italic>\n                    . While prior research has demonstrated that SATD can serve as a reliable indicator of technical debt and has extensively studied SATD in traditional (non-ML) software, little attention has been given to this issue in the context of ML. This article aims to investigate the occurrence of SATD in ML code by analyzing 318 open source ML projects across five domains, along with 318 non-ML projects. We detected SATD in source code comments in various snapshots of the studied projects, conducted a manual analysis of a sample of the identified SATD to comprehend the nature of technical debt in the ML code, and performed a survival analysis of the SATD to understand the evolution dynamics of such debts. Our analyses yielded the following observations: (i) ML projects have a median percentage of SATD that is twice that of non-ML projects. (ii) ML pipeline stages for\n                    <jats:italic toggle=\"yes\">data preprocessing<\/jats:italic>\n                    and\n                    <jats:italic toggle=\"yes\">model generation logic<\/jats:italic>\n                    are more susceptible to debt than\n                    <jats:italic toggle=\"yes\">model validation<\/jats:italic>\n                    and\n                    <jats:italic toggle=\"yes\">deployment<\/jats:italic>\n                    stages. (iii) SATDs appear in ML projects earlier in the development process compared to non-ML projects. (iv) Long-lasting SATDs are typically introduced during extensive code changes that span multiple files, which exhibit low complexity.\n                  <\/jats:p>\n                  <jats:p>Our research contributes to the understanding of technical debt in an ML context and underscores the need for targeted debt management strategies. This contribution is particularly relevant for developers and stakeholders in ML projects by aiding them in identifying and addressing technical debt proactively and paving the way for future research in developing automated tools and methodologies for managing SATD in an ML environment.<\/jats:p>","DOI":"10.1145\/3785001","type":"journal-article","created":{"date-parts":[[2025,12,15]],"date-time":"2025-12-15T18:28:43Z","timestamp":1765823323000},"page":"1-41","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["An Empirical Study of Self-Admitted Technical Debt in Machine Learning Software"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3552-9460","authenticated-orcid":false,"given":"Aaditya","family":"Bhatia","sequence":"first","affiliation":[{"name":"SAIL, Queen\u2019s University, Kingston, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5704-4173","authenticated-orcid":false,"given":"Foutse","family":"Khomh","sequence":"additional","affiliation":[{"name":"Polytechnique Montreal, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7213-4006","authenticated-orcid":false,"given":"Bram","family":"Adams","sequence":"additional","affiliation":[{"name":"Queen\u2019s University, Kingston, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7749-5513","authenticated-orcid":false,"given":"Ahmed","family":"Hassan","sequence":"additional","affiliation":[{"name":"Queen\u2019s University, Kingston, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,7,17]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aaa8415"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-022-10282-8"},{"key":"e_1_3_2_4_2","first-page":"51","volume-title":"Proceedings of the 2021 IEEE\/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)","author":"Idowu S.","year":"2021","unstructured":"S. Idowu, D. Str\u00fcber, and T. Berger. 2021. Asset management in machine learning: A survey. In Proceedings of the 2021 IEEE\/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 51\u201360."},{"key":"e_1_3_2_5_2","first-page":"2020","article-title":"Continuous delivery for machine learning","volume":"9","author":"Sato D.","year":"2019","unstructured":"D. Sato, A. Wider, and C. Windheuser. 2019. Continuous delivery for machine learning. Martin Fowler 9 (2019), 2020.","journal-title":"Martin Fowler"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.5555\/3586589.3586815"},{"issue":"9","key":"e_1_3_2_7_2","first-page":"1857","article-title":"How does machine learning change software development practices","volume":"47","author":"Wan Z.","year":"2019","unstructured":"Z. Wan, X. Xia, D. Lo, and G. C. Murphy. 2019. How does machine learning change software development practices? IEEE Transactions on Software Engineering 47, 9 (2019), 1857\u20131871.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_8_2","first-page":"2503","volume-title":"Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS \u201915)","volume":"2","author":"Sculley D.","year":"2015","unstructured":"D. Sculley, G. Holt, D. Golovin, E. Davydov, T. Phillips, D. Ebner, V. Chaudhary, M. Young, J.-F. Crespo, and D. Dennison. 2015. Hidden technical debt in machine learning systems. In Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS \u201915), Vol. 2. MIT Press, Cambridge, MA, USA, 2503\u20132511."},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","first-page":"734","DOI":"10.1145\/3540250.3549088","volume-title":"Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering","author":"O\u2019Brien D.","year":"2022","unstructured":"D. O\u2019Brien, S. Biswas, S. Imtiaz, R. Abdalkareem, E. Shihab, and H. Rajan. 2022. 23 shades of self-admitted technical debt: An empirical study on machine learning software. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 734\u2013746."},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/157710.157715"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/1882362.1882373"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/2901739.2901742"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/1985362.1985371"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-014-9351-7"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.02.056"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387906.3388630"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2012.130"},{"issue":"2","key":"e_1_3_2_18_2","first-page":"1","article-title":"An exploratory study on the introduction and removal of different types of technical debt in deep learning frameworks","volume":"26","author":"Liu J.","year":"2021","unstructured":"J. Liu, Q. Huang, X. Xia, E. Shihab, D. Lo, and S. Li. 2021. An exploratory study on the introduction and removal of different types of technical debt in deep learning frameworks. Empirical Software Engineering 26, 2 (2021), 1\u201336.","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_2_19_2","first-page":"179","volume-title":"Proceedings of the 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER)","volume":"1","author":"Wehaibi S.","year":"2016","unstructured":"S. Wehaibi, E. Shihab, and L. Guerrouj. 2016. Examining the impact of self-admitted technical debt on software quality. In Proceedings of the 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Vol. 1. IEEE, 179\u2013188."},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2012.129"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2012.167"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/2507288.2507326"},{"key":"e_1_3_2_23_2","unstructured":"E. Nascimento A. Nguyen-Duc I. Sundb\u00f8 and T. Conte. 2020. Software engineering for artificial intelligence and machine learning software: A systematic literature review. arXiv:2011.03751. Retrieved from https:\/\/arxiv.org\/abs\/2011.03751"},{"key":"e_1_3_2_24_2","first-page":"287","volume-title":"Proceedings of the 2021 IEEE\/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)","author":"Moin A.","year":"2021","unstructured":"A. Moin. 2021. Data analytics and machine learning methods, techniques and tool for model-driven engineering of smart IoT services. In Proceedings of the 2021 IEEE\/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). IEEE, 287\u2013292."},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2876537"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377814.3381714"},{"key":"e_1_3_2_27_2","first-page":"1","volume-title":"Proceedings of the 2020 IEEE-HYDCON","author":"Singla K.","year":"2020","unstructured":"K. Singla, T. Vinayak, A. Arpitha, C. Naik, and J. Bose. 2020. Story and task issue analysis for agile machine learning projects. In Proceedings of the 2020 IEEE-HYDCON. IEEE, 1\u20134."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3543847"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379597.3387479"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2014.31"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/WICSA-ECSA.212.17"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2015.10.008"},{"key":"e_1_3_2_33_2","first-page":"238","volume-title":"Proceedings of the 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME)","author":"Maldonado E. D. S.","year":"2017","unstructured":"E. D. S. Maldonado, R. Abdalkareem, E. Shihab, and A. Serebrenik. 2017. An empirical study on the removal of self-admitted technical debt. In Proceedings of the 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 238\u2013248."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2022.106855"},{"key":"e_1_3_2_35_2","first-page":"49","volume-title":"Proceedings of the 2019 10th International Workshop on Empirical Software Engineering in Practice (IWESEP)","author":"Washizaki H.","year":"2019","unstructured":"H. Washizaki, H. Uchida, F. Khomh, and Y.-G. Gu\u00e9h\u00e9neuc. 2019. Studying software engineering patterns for designing machine learning systems. In Proceedings of the 2019 10th International Workshop on Empirical Software Engineering in Practice (IWESEP). IEEE, 49\u2013495."},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2020.2993662"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3487043"},{"key":"e_1_3_2_38_2","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1145\/3183440.3183478","volume-title":"Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings","author":"Liu Z.","year":"2018","unstructured":"Z. Liu, Q. Huang, X. Xia, E. Shihab, D. Lo, and S. Li. 2018. SATD detector: A text-mining-based self-admitted technical debt detection tool. In Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings, 9\u201312."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2654244"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9522-4"},{"issue":"3","key":"e_1_3_2_41_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3324916","article-title":"Neural network-based detection of self-admitted technical debt: From performance to explainability","volume":"28","author":"Ren X.","year":"2019","unstructured":"X. Ren, Z. Xing, X. Xia, D. Lo, X. Wang, and J. Grundy. 2019. Neural network-based detection of self-admitted technical debt: From performance to explainability. ACM Transactions on Software Engineering and Methodology 28, 3 (2019), 1\u201345.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106270"},{"key":"e_1_3_2_43_2","first-page":"216","volume-title":"Proceedings of the 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME),","author":"Zampetti F.","year":"2017","unstructured":"F. Zampetti, C. Noiseux, G. Antoniol, F. Khomh, and M. Di Penta. 2017. Recommending when design technical debt should be self-admitted. In Proceedings of the 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME), 216\u2013226."},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2012.12.052"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11831-019-09344-w"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1017\/ATSIP.2015.22"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-14596-5"},{"issue":"9","key":"e_1_3_2_49_2","first-page":"1","article-title":"Applications of artificial intelligence in machine learning: Review and prospect","volume":"115","author":"Das S.","year":"2015","unstructured":"S. Das, A. Dey, A. Pal, and N. Roy. 2015. Applications of artificial intelligence in machine learning: Review and prospect. International Journal of Computer Applications 115, 9 (2015), 1\u201311.","journal-title":"International Journal of Computer Applications"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2018.01.002"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2020.113816"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/JAS.2020.1003021"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3234150"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/2597073.2597074"},{"key":"e_1_3_2_55_2","first-page":"84","volume-title":"Proceedings of the 2017 32nd IEEE\/ACM International Conference on Automated Software Engineering (ASE)","author":"Mirhosseini S.","year":"2017","unstructured":"S. Mirhosseini and C. Parnin. 2017. Can automated pull requests encourage software developers to upgrade out-of-date dependencies? In Proceedings of the 2017 32nd IEEE\/ACM International Conference on Automated Software Engineering (ASE). IEEE, 84\u201394."},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106274"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09902-y"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR52588.2021.00074"},{"key":"e_1_3_2_59_2","first-page":"254","volume-title":"Proceedings of the 2021 IEEE\/ACM 18th International Conference on Mining Software Repositories (MSR)","author":"Alfadel M.","year":"2021","unstructured":"M. Alfadel, D. E. Costa, E. Shihab, and M. Mkhallalati. 2021. On the use of dependabot security pull requests. In Proceedings of the 2021 IEEE\/ACM 18th International Conference on Mining Software Repositories (MSR). IEEE, 254\u2013265."},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380408"},{"issue":"6","key":"e_1_3_2_61_2","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1007\/s10664-023-10382-z","article-title":"Studying the characteristics of AIOps projects on GitHub","volume":"28","author":"Aghili R.","year":"2023","unstructured":"R. Aghili, H. Li, and F. Khomh. 2023. Studying the characteristics of AIOps projects on GitHub. Empirical Software Engineering 28, 6 (2023), 143.","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-018-9660-3"},{"issue":"3","key":"e_1_3_2_63_2","doi-asserted-by":"crossref","first-page":"1018","DOI":"10.1109\/TSE.2020.3010045","article-title":"Quantifying, characterizing, and mitigating flakily covered program elements","volume":"48","author":"Vysali S.","year":"2020","unstructured":"S. Vysali, S. McIntosh, and B. Adams. 2020. Quantifying, characterizing, and mitigating flakily covered program elements. IEEE Transactions on Software Engineering 48, 3 (2020), 1018\u20131029.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2015.12"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33876-2_7"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-021-09564-z"},{"issue":"8","key":"e_1_3_2_67_2","doi-asserted-by":"crossref","first-page":"3159","DOI":"10.1109\/TSE.2021.3082074","article-title":"In search of socio-technical congruence: A large-scale longitudinal study","volume":"48","author":"Mauerer W.","year":"2021","unstructured":"W. Mauerer, M. Joblin, D. A. Tamburri, C. Paradis, R. Kazman, and S. Apel. 2021. In search of socio-technical congruence: A large-scale longitudinal study. IEEE Transactions on Software Engineering 48, 8 (2021), 3159\u20133184.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_68_2","first-page":"171","volume-title":"Proceedings of the 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER)","author":"Morales R.","year":"2015","unstructured":"R. Morales, S. McIntosh, and F. Khomh. 2015. Do code review practices impact design quality? A case study of the qt, VTK, and ITK projects. In Proceedings of the 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER), 171\u2013180."},{"issue":"4","key":"e_1_3_2_69_2","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1007\/s10664-024-10456-6","article-title":"The broken windows theory applies to technical debt","volume":"29","author":"Lev\u00e9n W.","year":"2024","unstructured":"W. Lev\u00e9n, H. Broman, T. Besker, and R. Torkar. 2024. The broken windows theory applies to technical debt. Empirical Software Engineering 29, 4 (2024), 73.","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_2_70_2","unstructured":"A. Nguyen-Duc. 2017. The impact of software complexity on cost and quality\u2013A comparative analysis between open source and proprietary software. arXiv:1712.00675. Retrieved from https:\/\/arxiv.org\/abs\/1712.00675"},{"key":"e_1_3_2_71_2","first-page":"54","volume-title":"Proceedings of the 2020 IEEE 20th International Working Conference on Source Code Analysis and Manipulation (SCAM),","author":"Maipradit R.","year":"2020","unstructured":"R. Maipradit, B. Lin, C. Nagy, G. Bavota, M. Lanza, H. Hata, and K. Matsumoto. 2020. Automated identification of on-hold self-admitted technical debt. In Proceedings of the 2020 IEEE 20th International Working Conference on Source Code Analysis and Manipulation (SCAM), 54\u201364."},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.5555\/2755629"},{"key":"e_1_3_2_73_2","first-page":"99","volume-title":"Proceedings of the 2008 International Working Conference on Mining Software Repositories","author":"Hindle A.","year":"2008","unstructured":"A. Hindle, D. M. German, and R. Holt. 2008. What do large commits tell us? A taxonomical study of large commits. In Proceedings of the 2008 International Working Conference on Mining Software Repositories, 99\u2013108."},{"key":"e_1_3_2_74_2","first-page":"1028","volume-title":"Proceedings of the 2016 IEEE\/ACM 38th International Conference on Software Engineering (ICSE)","author":"Kononenko O.","year":"2016","unstructured":"O. Kononenko, O. Baysal, and M. W. Godfrey. 2016. Code review quality: How developers see it. In Proceedings of the 2016 IEEE\/ACM 38th International Conference on Software Engineering (ICSE), 1028\u20131038."},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2900245"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2024.107450"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1097\/00001648-199609000-00030"},{"key":"e_1_3_2_78_2","volume-title":"Survival Analysis","author":"Miller R. G.","year":"2011","unstructured":"R. G. Miller. Jr. 2011. Survival Analysis, Vol. 66. John Wiley and Sons."},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1958.10501452"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2831232"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2584050"},{"key":"e_1_3_2_82_2","volume-title":"A Value for n-Person Games","author":"Shapley L. S.","year":"2016","unstructured":"L. S. Shapley. 2016. A Value for n-Person Games. Princeton University Press."},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09854-3"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1145\/2597073.2597076"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1109\/WCRE.2012.32"},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377815.3381377"},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1109\/MTD.2014.9"},{"key":"e_1_3_2_88_2","first-page":"9","volume-title":"Proceedings of the 2015 IEEE 7th International Workshop on Managing Technical Debt (MTD)","author":"Maldonado E. D S.","year":"2015","unstructured":"E. D S. Maldonado and E. Shihab. 2015. Detecting and quantifying different types of self-admitted technical debt. In Proceedings of the 2015 IEEE 7th International Workshop on Managing Technical Debt (MTD). IEEE, 9\u201315."},{"key":"e_1_3_2_89_2","first-page":"7","volume-title":"Proceedings of the 2018 9th International Workshop on Empirical Software Engineering in Practice (IWESEP)","author":"Wattanakriengkrai S.","year":"2018","unstructured":"S. Wattanakriengkrai, R. Maipradit, H. Hata, M. Choetkiertikul, T. Sunetnanta, and K. Matsumoto. 2018. Identifying design and requirement self-admitted technical debt using N-gram IDF. In Proceedings of the 2018 9th International Workshop on Empirical Software Engineering in Practice (IWESEP). IEEE, 7\u201312."},{"key":"e_1_3_2_90_2","doi-asserted-by":"crossref","first-page":"1489","DOI":"10.1061\/41020(339)151","volume-title":"Proceedings of the Construction Research Congress 2009: Building a Sustainable Future","author":"Hallowell M. R.","year":"2009","unstructured":"M. R. Hallowell. 2009. Techniques to minimize bias when using the Delphi method to quantify construction safety and health risks. In Proceedings of the Construction Research Congress 2009: Building a Sustainable Future, 1489\u20131498."},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2962027"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2014.12.027"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2018.05.010"},{"key":"e_1_3_2_94_2","volume-title":"Refactoring: improving the Design of Existing Code","author":"Fowler M.","year":"2018","unstructured":"M. Fowler. 2018. Refactoring: improving the Design of Existing Code. Addison-Wesley Professional."},{"issue":"13","key":"e_1_3_2_95_2","doi-asserted-by":"crossref","first-page":"15137165","DOI":"10.3390\/app15137165","article-title":"A scoping review and assessment framework for technical debt in the development and operation of AI\/ML competition platforms","volume":"15","author":"Sklavenitis D.","year":"2025","unstructured":"D. Sklavenitis and D. Kalles. 2025. A scoping review and assessment framework for technical debt in the development and operation of AI\/ML competition platforms. Applied Sciences 15, 13 (2025), 15137165.","journal-title":"Applied Sciences"},{"key":"e_1_3_2_96_2","doi-asserted-by":"crossref","first-page":"107678","DOI":"10.1016\/j.infsof.2025.107678","article-title":"Classification and challenges of non-functional requirements in ML-enabled systems: A systematic literature review","author":"De Martino V.","year":"2025","unstructured":"V. De Martino and F. Palomba. 2025. Classification and challenges of non-functional requirements in ML-enabled systems: A systematic literature review. Information and Software Technology 181 (2025) 107678.","journal-title":"Information and Software Technology"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1145\/3688671.3688783"},{"key":"e_1_3_2_98_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09901-z"},{"key":"e_1_3_2_99_2","first-page":"558","volume-title":"Proceedings of the 2023 IEEE International Conference on Software Maintenance and Evolution (ICSME)","author":"Li Y.","year":"2023","unstructured":"Y. Li, M. Soliman, P. Avgeriou, and M. Van Ittersum. 2023. DebtViz: A tool for identifying, measuring, visualizing, and monitoring self-admitted technical debt. In Proceedings of the 2023 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 558\u2013562."},{"key":"e_1_3_2_100_2","volume-title":"Decision-Making Using Service Level Objectives","author":"Hidalgo A.","year":"2020","unstructured":"A. Hidalgo. 2020. Decision-Making Using Service Level Objectives, Vol. 30. Usenix Patrons."},{"key":"e_1_3_2_101_2","first-page":"84","volume-title":"Proceedings of the International Conference on Agile Software Development","author":"Tonin G. S.","year":"2017","unstructured":"G. S. Tonin, A. Goldman, C. Seaman, and D. Pina. 2017. Effects of technical debt awareness: A classroom study. In Proceedings of the International Conference on Agile Software Development. Springer International Publishing, 84\u2013100."},{"key":"e_1_3_2_102_2","first-page":"99","volume-title":"Proceedings of the 2021 IEEE\/ACM International Conference on Technical Debt (TechDebt)","author":"Crespo Y.","year":"2021","unstructured":"Y. Crespo, A. Gonzalez-Escribano, and M. Piattini. 2021. Carrot and stick approaches revisited when managing technical debt in an educational context. In Proceedings of the 2021 IEEE\/ACM International Conference on Technical Debt (TechDebt). IEEE, 99\u2013108."}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3785001","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T16:04:22Z","timestamp":1784304262000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3785001"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,17]]},"references-count":101,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2026,8,31]]}},"alternative-id":["10.1145\/3785001"],"URL":"https:\/\/doi.org\/10.1145\/3785001","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,17]]},"assertion":[{"value":"2023-12-03","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-13","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-07-17","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}