{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T19:39:29Z","timestamp":1771616369866,"version":"3.50.1"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2025,12,31]]},"abstract":"<jats:p>Rapid advancements in Machine Learning (ML) introduce unique maintainability and scalability challenges. Our research addresses the evolving challenges and identifies ML maintainability and scalability solutions by conducting a thorough literature review of over 17,000 papers, ultimately refining our focus to 124 relevant sources that meet our stringent selection criteria. We present a catalogue of 41 Maintainability and 13 Scalability challenges and solutions across Data, Model Engineering and the overall development of ML applications and systems. This study equips practitioners with insights on building robust ML applications, laying the groundwork for future research on improving ML system robustness at different workflow stages.<\/jats:p>","DOI":"10.1145\/3736751","type":"journal-article","created":{"date-parts":[[2025,5,22]],"date-time":"2025-05-22T07:17:51Z","timestamp":1747898271000},"page":"1-36","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Maintainability and Scalability in Machine Learning: Challenges and Solutions"],"prefix":"10.1145","volume":"57","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-8508-2978","authenticated-orcid":false,"given":"Karthik","family":"Shivashankar","sequence":"first","affiliation":[{"name":"Department of Informatics, University of Oslo","place":["Oslo, Norway"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1639-1424","authenticated-orcid":false,"given":"Ghadi","family":"Al Hajj","sequence":"additional","affiliation":[{"name":"Department of Informatics, University of Oslo","place":["Oslo, Norway"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0669-8687","authenticated-orcid":false,"given":"Antonio","family":"Martini","sequence":"additional","affiliation":[{"name":"Informatics, University of Oslo","place":["Oslo, Norway"]}]}],"member":"320","published-online":{"date-parts":[[2025,7,12]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"crossref","unstructured":"Saleema Amershi Andrew Begel Christian Bird Robert DeLine Harald Gall Ece Kamar Nachiappan Nagappan Besmira Nushi and Thomas Zimmermann. 2019. Software engineering for machine learning: A case study. In 2019 IEEE\/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE 291\u2013300.","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA.2018.00018"},{"key":"e_1_3_3_4_2","article-title":"Architectures and optimization strategies for real-time machine learning recommendation systems: A systematic review of scalability challenges","author":"Bharti M.","year":"2025","unstructured":"M. Bharti. 2025. Architectures and optimization strategies for real-time machine learning recommendation systems: A systematic review of scalability challenges. International Journal of Scientific Research in Computer Science, Engineering and Information Technology (2025). Forthcoming article. Volume, number, and page information not yet available.","journal-title":"International Journal of Scientific Research in Computer Science, Engineering and Information Technology"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-023-10493-5"},{"key":"e_1_3_3_6_2","article-title":"ML Scalability","author":"AI Censius","year":"2025","unstructured":"Censius AI. 2025. ML Scalability. Censius AI Wiki, Retrieved from https:\/\/censius.ai\/wiki\/ml-scalability. Accessed: April 28, 2025.","journal-title":"Censius AI Wiki, Retrieved from"},{"key":"e_1_3_3_7_2","unstructured":"Conceptatech. 2023. The Importance of Scalability In Software Design. Retrieved from https:\/\/www.conceptatech.com\/blog\/importance-of-scalability-in-software-design. Accessed: March 17 2023. Original publication date not specified."},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.3390\/agronomy12030748"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3453478"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2008.01.006"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2021.3069039"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.111031"},{"key":"e_1_3_3_13_2","unstructured":"Ian Goodfellow Yoshua Bengio and Aaron Courville. 2016. Deep Learning. MIT Press. http:\/\/www.deeplearningbook.org"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.20982\/tqmp.08.1.p023"},{"key":"e_1_3_3_15_2","volume-title":"Designing Machine Learning Systems","author":"Huyen Chip","year":"2022","unstructured":"Chip Huyen. 2022. Designing Machine Learning Systems. O\u2019Reilly Media, Inc."},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/IEEESTD.1998.88278"},{"key":"e_1_3_3_17_2","unstructured":"InterviewNode. 2024. Architecting Scalable ML Systems: A Guide for Senior Engineers by InterviewNode. Retrieved from https:\/\/www.interviewnode.com\/. Accessed: December 17 2024. Original publication date not specified."},{"key":"e_1_3_3_18_2","unstructured":"Christian Kastner. 2025. Machine learning in production: From models to products. MIT Press."},{"key":"e_1_3_3_19_2","volume-title":"Procedures for Performing Systematic Reviews","author":"Kitchenham B.","year":"2004","unstructured":"B. Kitchenham. 2004. Procedures for Performing Systematic Reviews. Technical Report TR\/SE-0401. Keele University, Keele, UK."},{"key":"e_1_3_3_20_2","volume-title":"Proceedings of the 1st Workshop on Software Engineering for Responsible AI (SE4RAI\u201922)","author":"Kolltveit Ask Berstad","year":"2023","unstructured":"Ask Berstad Kolltveit and Jingyue Li. 2023. Operationalising machine learning models: A systematic literature review. In Proceedings of the 1st Workshop on Software Engineering for Responsible AI (SE4RAI\u201922). Page numbers not provided in the source. The workshop was in 2022, proceedings published in 2023."},{"key":"e_1_3_3_21_2","volume-title":"Content Analysis: An Introduction to its Methodology","author":"Krippendorff Klaus","year":"2018","unstructured":"Klaus Krippendorff. 2018. Content Analysis: An Introduction to its Methodology. Sage Publications."},{"key":"e_1_3_3_22_2","doi-asserted-by":"crossref","unstructured":"Marco Kuhrmann Daniel M\u00e9ndez Fern\u00e1ndez and Maya Daneva. 2017. On the pragmatic design of literature studies in software engineering: an experience-based guideline. Empirical Software Engineering 22 (2017) 2852\u20132891.","DOI":"10.1007\/s10664-016-9492-y"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.2307\/2529310"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477535"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-19034-7_14"},{"key":"e_1_3_3_26_2","unstructured":"Ruchika Malhotra and Anuradha Chug. 2012. Software maintainability prediction using machine learning algorithms. Software Engineering: An International Journal (SeiJ) 2 2 (2012)."},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1142\/S0218194016500431"},{"key":"e_1_3_3_28_2","unstructured":"Microsoft. 2024. Scalability - Engineering Fundamentals Playbook. https:\/\/microsoft.github.io\/code-with-engineeringplaybook\/non-functional-requirements\/scalability\/"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-65854-0_8"},{"key":"e_1_3_3_30_2","unstructured":"ML-Architects. 2024. System Design and Continuous Delivery. Retrieved from https:\/\/ml-architects.ch\/. Accessed: December 27 2024. Original publication date not specified."},{"key":"e_1_3_3_31_2","unstructured":"ml-ops.org. 2023. ml-ops.org. Retrieved from https:\/\/ml-ops.org\/. Accessed: March 17 2023. Original publication date not specified."},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA.2019.00030"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510209"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2019.8870157"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549088"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9498-0"},{"key":"e_1_3_3_37_2","doi-asserted-by":"crossref","unstructured":"Paraskumar Patel. 2023. Challenges and solutions in cross-platform ML systems integration. Journal of Artificial Intelligence and Cloud Computing 2 4 (2023) 1\u20135.","DOI":"10.47363\/JAICC\/2023(2)233"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/215206.215351"},{"key":"e_1_3_3_39_2","first-page":"2503","volume-title":"Advances in Neural Information Processing Systems 28","author":"Sculley D.","year":"2015","unstructured":"D. Sculley, Gary Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-Francois Crespo, and Dan Dennison. 2015. Hidden technical debt in machine learning systems. In Advances in Neural Information Processing Systems 28, C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett (Eds.). Curran Associates, Inc., 2503\u20132511."},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3382494.3410681"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA56994.2022.00018"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.4135\/9781473906907"},{"key":"e_1_3_3_43_2","article-title":"Overcoming Pitfalls in ML System Design","year":"2025","unstructured":"Trinetix. 2025. Overcoming Pitfalls in ML System Design. Trinetix Insights, Retrieved from https:\/\/www.trinetix.com\/. Accessed: January 21, 2025. Original publication date not specified.","journal-title":"Trinetix Insights, Retrieved from"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","unstructured":"Rajesh Vasa. 2025. Microservices vs. Monolithic Architectures: Strategic Trade-Offs in the AI Era. International Journal of Scientific Research in Computer Science Engineering and Information Technology 11 1 (February 2025) 2428\u20132436. 10.32628\/CSEIT251112269","DOI":"10.32628\/CSEIT251112269"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3173346"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC51774.2021.00207"},{"issue":"2","key":"e_1_3_3_47_2","first-page":"xiii\u2013xxiii","article-title":"Analysing the past to prepare for the future: Writing a literature review","volume":"26","author":"Webster J.","year":"2002","unstructured":"J. Webster and R. T. Watson. 2002. Analysing the past to prepare for the future: Writing a literature review. MIS Quarterly 26, 2 (2002), xiii\u2013xxiii.","journal-title":"MIS Quarterly"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/2601248.2601268"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2013.04.076"},{"key":"e_1_3_3_50_2","doi-asserted-by":"crossref","unstructured":"Jeff Jun Zhang Kang Liu Faiq Khalid Muhammad Abdullah Hanif Semeen Rehman Theocharis Theocharides Alessandro Artussi Muhammad Shafique and Siddharth Garg. 2019. Building robust machine learning systems: Current progress research challenges and opportunities. In Proceedings of the 56th Annual Design Automation Conference 2019. 1\u20134.","DOI":"10.1145\/3316781.3323472"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3736751","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,17]],"date-time":"2025-07-17T13:27:38Z","timestamp":1752758858000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3736751"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,12]]},"references-count":49,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2025,12,31]]}},"alternative-id":["10.1145\/3736751"],"URL":"https:\/\/doi.org\/10.1145\/3736751","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,12]]},"assertion":[{"value":"2024-01-17","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-03","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}