{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,27]],"date-time":"2026-04-27T14:59:24Z","timestamp":1777301964909,"version":"3.51.4"},"reference-count":94,"publisher":"Association for Computing Machinery (ACM)","issue":"11","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2026,8,30]]},"abstract":"<jats:p>The adoption of Machine and Deep Learning (ML\/DL) technologies introduces maintenance challenges, leading to Technical Debt (TD). Algorithm\u00a0Debt (AD) is a TD type that impacts the performance and scalability of ML\/DL systems. A review of 42 primary studies expanded AD\u2019s definition, uncovered its implicit presence, identified its smells, and highlighted future directions. These findings will guide an AD-focused study, enhancing the reliability of ML\/DL systems.<\/jats:p>","DOI":"10.1145\/3806391","type":"journal-article","created":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T11:20:24Z","timestamp":1775128824000},"page":"1-35","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["A Survey of Algorithm Debt in Machine and Deep Learning Systems: Definition, Smells, and Future Work"],"prefix":"10.1145","volume":"58","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8494-6885","authenticated-orcid":false,"given":"Emmanuel","family":"Simon","sequence":"first","affiliation":[{"name":"School of Computing, Australian National University","place":["Canberra, Australia"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7702-0718","authenticated-orcid":false,"given":"Chirath","family":"Hettiarachchi","sequence":"additional","affiliation":[{"name":"School of Computing, Australian National University","place":["Canberra, Australia"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4505-6257","authenticated-orcid":false,"given":"Fatemeh","family":"Fard","sequence":"additional","affiliation":[{"name":"Computer Science Department, University of British Columbia","place":["Kelowna, Canada"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4242-2725","authenticated-orcid":false,"given":"Alex","family":"Potanin","sequence":"additional","affiliation":[{"name":"School of Computing, Australian National University","place":["Canberra, Australia"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4195-1641","authenticated-orcid":false,"given":"Hanna","family":"Suominen","sequence":"additional","affiliation":[{"name":"School of Computing, Australian National University","place":["Canberra, Australia"]},{"name":"Department of Computing, University of Turku","place":["Canberra, Australia"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,4,27]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Adekunle Akinjobi Ajibode Yvon Apedo and Temitope Ajibode. 2024. Systematic literature review on forecasting and prediction of technical debt evolution. arXiv:2406.12026. Retrieved from https:\/\/arxiv.org\/abs\/2406.12026 Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:270562091"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524843.3528097"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13369-023-08596-w"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2015.10.008"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/MTD.2014.9"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jik.2024.100637"},{"key":"e_1_3_2_8_2","volume-title":"Proceedings of the Dagstuhl Reports","author":"Avgeriou Paris","year":"2016","unstructured":"Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman. 2016. Managing technical debt in software engineering (dagstuhl seminar 16162). In Proceedings of the Dagstuhl Reports. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-021-10081-7"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2022.2138826"},{"key":"e_1_3_2_11_2","article-title":"Investigate, identify and estimate the technical debt: A systematic mapping study","author":"BenIdris Mrwan","year":"2018","unstructured":"Mrwan BenIdris. 2018. Investigate, identify and estimate the technical debt: A systematic mapping study. Econometrics: Econometric and Statistical Methods - Special Topics eJournal (2018). Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:53070347","journal-title":"Econometrics: Econometric and Statistical Methods - Special Topics eJournal"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2017.09.025"},{"key":"e_1_3_2_13_2","unstructured":"Aaditya Bhatia Foutse Khomh Bram Adams and Ahmed E. Hassan. 2023. An empirical study of self-admitted technical debt in machine learning software. ACM Transactions on Software Engineering and Methodology (2023). Retrieved from https:\/\/arxiv.org\/abs\/2311.12019"},{"issue":"1","key":"e_1_3_2_14_2","article-title":"An empirical study of self-admitted technical debt in machine learning software","volume":"1","author":"Bhatia Aaditya","year":"2024","unstructured":"Aaditya Bhatia, Foutse Khomh, Bram Adams, and Ahmed E. Hassan. 2024. An empirical study of self-admitted technical debt in machine learning software. ACM Transaction on Software Engineering Methodologies 1, 1 (2024).","journal-title":"ACM Transaction on Software Engineering Methodologies"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2023.107375"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409704"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/TechDebt52882.2021.00016"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1191\/1478088706qp063oa"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2017.8258038"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2006.07.009"},{"key":"e_1_3_2_21_2","unstructured":"Nicol\u00e1s Cardozo Ivana Dusparic and Christian Cabrera. 2023. Prevalence of code smells in reinforcement learning projects. arXiv:2303.10236. Retrieved from https:\/\/arxiv.org\/abs\/2303.10236"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-32489-6_13"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICGCIoT.2018.8753081"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/SPARC61891.2024.10829223"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","unstructured":"Pierre-Olivier C\u00f4t\u00e9 Amin Nikanjam Rached Bouchoucha Ilan Basta Mouna Abidi and Foutse Khomh. 2023. Quality issues in machine learning software systems. Empirical Software Engineering 29 6 (2023).","DOI":"10.1007\/s10664-024-10536-7"},{"key":"e_1_3_2_27_2","volume-title":"Acquiring Maintainable AI-Enabled Systems","author":"Cruickshank MAJ Iain","year":"2023","unstructured":"MAJ Iain Cruickshank and MAJ Shane Kohtz. 2023. Acquiring Maintainable AI-Enabled Systems. Technical Report. Acquisition Research Program."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/157710.157715"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclepro.2019.01.152"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0031619"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/MTD.2012.6225993"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSTARS.2025.3527898"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.111031"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.5555\/3086952"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3644384.3644475"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-023-00410-1"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380395"},{"key":"e_1_3_2_38_2","first-page":"84","volume-title":"Proceedings of the Joint of the 4th International Workshop on Quantitative Approaches to Software Quality, QuASoQ 2016 and 1st International Workshop on Technical Debt Analytics, TDA 2016","author":"Izurieta Clemente","year":"2016","unstructured":"Clemente Izurieta, Ipek Ozkaya, Carolyn Seaman, Philippe Kruchten, Robert Nord, Will Snipes, and Paris Avgeriou. 2016. Perspectives on managing technical debt: A transition point and roadmap from Dagstuhl. In Proceedings of the Joint of the 4th International Workshop on Quantitative Approaches to Software Quality, QuASoQ 2016 and 1st International Workshop on Technical Debt Analytics, TDA 2016. 84\u201387."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.110935"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-024-10521-0"},{"issue":"1","key":"e_1_3_2_41_2","first-page":"155","article-title":"AI in finance disruptive technologies and emerging opportunities","volume":"3","author":"Khan AKM Kamruzzaman","year":"2024","unstructured":"AKM Kamruzzaman Khan. 2024. AI in finance disruptive technologies and emerging opportunities. Journal of Artificial Intelligence General science (JAIGS) 3, 1 (2024), 155\u2013170.","journal-title":"Journal of Artificial Intelligence General science (JAIGS)"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2013.07.010"},{"key":"e_1_3_2_43_2","volume-title":"Guidelines for Performing Systematic Literature Reviews in Software Engineering (ebse Technical Report Version 2.3, ebse-2007-01)","author":"Kitchenham B.","year":"2007","unstructured":"B. Kitchenham and S. Charters. 2007. Guidelines for Performing Systematic Literature Reviews in Software Engineering (ebse Technical Report Version 2.3, ebse-2007-01). Technical Report. Technical report, Keele University, University of Durham, Keele, United Kingdom."},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2004.1317449"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3165938"},{"key":"e_1_3_2_46_2","volume-title":"Standard Quality Assessment Criteria for Evaluating Primary Research Papers from a Variety of Fields","author":"Kmet Leanne M.","year":"2004","unstructured":"Leanne M. Kmet, Robert C. Lee, and Linda S. Cook. 2004. Standard Quality Assessment Criteria for Evaluating Primary Research Papers from a Variety of Fields. Technical Report HTA Initiative No. 13. Alberta Heritage Foundation for Medical Research, Edmonton, Alberta, Canada. Retrieved from https:\/\/ia601308.us.archive.org\/33\/items\/standardqualitya00kmet_0\/standardqualitya00kmet_0.pdf"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110827"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00279"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2023.107367"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377815.3381377"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09917-5"},{"key":"e_1_3_2_52_2","volume-title":"The Ultimate Guide to Model Retraining","author":"Liugi Patruno","year":"2019","unstructured":"Patruno Liugi. 2019. The Ultimate Guide to Model Retraining. Retrieved from https:\/\/mlinproduction.com\/model-retraining\/"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2023.107326"},{"issue":"2","key":"e_1_3_2_54_2","first-page":"20","article-title":"Qualitative content analysis","volume":"1","author":"Mayring Philipp","year":"2000","unstructured":"Philipp Mayring. 2000. Qualitative content analysis. Forum: Qualitative Social Research 1, 2 (2000), 20. Retrieved from https:\/\/www.qualitative-research.net\/index.php\/fqs\/article\/view\/1089","journal-title":"Forum: Qualitative Social Research"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1186\/2046-4053-4-1"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-023-10291-1"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2017.30"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106294"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA.2019.00030"},{"key":"e_1_3_2_60_2","unstructured":"Tolga Muratda\u011f\u0131. 2024. Identifying technical debt and tools for technical debt management in software development. Middle East Technical University."},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510209"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2023.111860"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME52107.2021.00036"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2023.3237460"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549088"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533378"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3182532"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME58944.2024.00043"},{"issue":"1","key":"e_1_3_2_69_2","first-page":"b92","article-title":"Guidance on the conduct of narrative synthesis in systematic reviews","volume":"1","author":"Popay Jennie","year":"2006","unstructured":"Jennie Popay, Helen Roberts, Amanda Sowden, Mark Petticrew, Lisa Arai, Mark Rodgers, Nicky Britten, Katrina Roen, Steven Duffy, et\u00a0al. 2006. Guidance on the conduct of narrative synthesis in systematic reviews. A Product from the ESRC Methods Programme Version 1, 1 (2006), b92.","journal-title":"A Product from the ESRC Methods Programme Version"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2014.31"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2024.112151"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-008-9102-8"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.37190\/e-Inf240101"},{"key":"e_1_3_2_74_2","article-title":"Hidden technical debt in machine learning systems","volume":"28","author":"Sculley David","year":"2015","unstructured":"David Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-Francois Crespo, and Dan Dennison. 2015. Hidden technical debt in machine learning systems. Advances in Neural Information Processing Systems 28 (2015), 2503\u20132511.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-022-00358-6"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA56994.2022.00018"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion66252.2025.00066"},{"key":"e_1_3_2_78_2","volume-title":"Proceedings of the Registered Report Track at the 40th IEEE International Conference on Software Maintenance and Evolution","author":"Simon Emmanuel Iko-Ojo","year":"2024","unstructured":"Emmanuel Iko-Ojo Simon, Chirath Hettiarachchi, Alex Potanin, Hanna Suominen, and Fatemeh Fard. 2024. Automated detection of algorithm debt in deep learning frameworks: An empirical study. In Proceedings of the Registered Report Track at the 40th IEEE International Conference on Software Maintenance and Evolution. arXiv:1409.0473. Retrieved from https:\/\/arxiv.org\/abs\/1701.00133"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/CAIN58948.2023.00020"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2017.41"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00033"},{"key":"e_1_3_2_82_2","first-page":"1","volume-title":"Proceedings of the 2021 IEEE\/ACM 1st Workshop on AI Engineering-Software Engineering for AI","author":"Oort Bart Van","year":"2021","unstructured":"Bart Van Oort, Lu\u00eds Cruz, Maur\u00edcio Aniche, and Arie Van Deursen. 2021. The prevalence of code smells in machine learning projects. In Proceedings of the 2021 IEEE\/ACM 1st Workshop on AI Engineering-Software Engineering for AI. IEEE, 1\u20138."},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524842.3528455"},{"key":"e_1_3_2_84_2","doi-asserted-by":"crossref","unstructured":"Roberto Verdecchia Kevin Maggi Leonardo Scommegna and Enrico Vicario. 2024. Technical debt in microservices: A mixed-method case study. In European Conference on Software Architecture. Springer 217\u2013236.","DOI":"10.1007\/978-3-031-66326-0_14"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00136"},{"key":"e_1_3_2_86_2","first-page":"179","volume-title":"Proceedings of the International Conference on Mining Software Repositories","author":"Vidoni Melina","year":"2021","unstructured":"Melina Vidoni. 2021. Self-admitted technical debt in R packages: An exploratory study. In Proceedings of the International Conference on Mining Software Repositories. IEEE, 179\u2013189."},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2022.111265"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2024.112053"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1109\/INDIN51400.2023.10217843"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40593-023-00342-5"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1145\/2601248.2601268"},{"key":"e_1_3_2_92_2","volume-title":"Issues that Lead to Code Technical Debt in Machine Learning Systems","author":"Ximenes Rodrigo Galdino","year":"2024","unstructured":"Rodrigo Galdino Ximenes. 2024. Issues that Lead to Code Technical Debt in Machine Learning Systems. Master\u2019s Dissertation. Pontifical Catholic University of Rio de Janeiro (PUC-Rio)."},{"key":"e_1_3_2_93_2","volume-title":"Case Study Research and Applications: Design and Methods (6th. ed.)","author":"Yin Robert K.","year":"2018","unstructured":"Robert K. Yin. 2018. Case Study Research and Applications: Design and Methods (6th. ed.). SAGE Publications, Thousand Oaks, CA. Retrieved from https:\/\/us.sagepub.com\/en-us\/nam\/case-study-research-and-applications\/book250150"},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.1145\/3522664.3528620"},{"key":"e_1_3_2_95_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2016.031"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3806391","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,27]],"date-time":"2026-04-27T14:18:04Z","timestamp":1777299484000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3806391"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,27]]},"references-count":94,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2026,8,30]]}},"alternative-id":["10.1145\/3806391"],"URL":"https:\/\/doi.org\/10.1145\/3806391","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,27]]},"assertion":[{"value":"2025-05-08","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-24","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-27","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}