{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T02:07:27Z","timestamp":1782439647560,"version":"3.54.5"},"reference-count":153,"publisher":"Association for Computing Machinery (ACM)","issue":"10","license":[{"start":{"date-parts":[[2025,5,6]],"date-time":"2025-05-06T00:00:00Z","timestamp":1746489600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2025,10,31]]},"abstract":"<jats:p>Emerging from the software crisis of the 1960s, conventional software systems have vastly improved through Software Engineering (SE) practices. Simultaneously, Artificial Intelligence (AI) endeavors to augment or replace human decision-making. In the contemporary landscape, Machine Learning (ML), a subset of AI, leverages extensive data from diverse sources, fostering the development of ML-enabled (intelligent) software systems. While ML is increasingly utilized in conventional software development, the integration of SE practices in developing ML-enabled systems, especially across typical Software Development Life Cycle (SDLC) phases and methodologies in the post-2010 Deep Learning (DL) era, remains underexplored. Our survey of existing literature unveils insights into current practices, emphasizing the interdisciplinary collaboration challenges of developing ML-enabled software, including data quality, ethics, explainability, continuous monitoring and adaptation, and security. The study underscores the imperative for ongoing research and development with focus on data-driven hypotheses, non-functional requirements, established design principles, ML-first integration, automation, specialized testing, and use of agile methods.<\/jats:p>","DOI":"10.1145\/3731597","type":"journal-article","created":{"date-parts":[[2025,4,19]],"date-time":"2025-04-19T10:12:06Z","timestamp":1745057526000},"page":"1-35","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["State-of-the-Art and Challenges of Engineering ML- Enabled Software Systems in the Deep Learning Era"],"prefix":"10.1145","volume":"57","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6760-690X","authenticated-orcid":false,"given":"Gebremariam","family":"Assres","sequence":"first","affiliation":[{"name":"Kristiania University of Applied Sciences, Oslo, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4032-7465","authenticated-orcid":false,"given":"Guru","family":"Bhandari","sequence":"additional","affiliation":[{"name":"Kristiania University of Applied Sciences, Oslo, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1980-6875","authenticated-orcid":false,"given":"Andrii","family":"Shalaginov","sequence":"additional","affiliation":[{"name":"Kristiania University of Applied Sciences, Oslo, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2026-4551","authenticated-orcid":false,"given":"Tor-Morten","family":"Gronli","sequence":"additional","affiliation":[{"name":"Kristiania University of Applied Sciences, Oslo, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2578-5580","authenticated-orcid":false,"given":"Gheorghita","family":"Ghinea","sequence":"additional","affiliation":[{"name":"Computer Science, Brunel University London, London, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,5,6]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/CCCI49893.2020.9256507"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/IOTSMS48152.2019.8939265"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3064867"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2018.10.006"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.rser.2020.109899"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA.2018.00018"},{"key":"e_1_3_2_9_2","volume-title":"CEUR Workshop Proceedings","volume":"3144","author":"Ayala Claudia","year":"2022","unstructured":"Claudia Ayala, Besim Bilalli, Cristina G\u00f3mez, and Silverio Mart\u00ednez-Fern\u00e1ndez. 2022. DOGO4ML: Development, operation and data governance for ML-based software systems. In CEUR Workshop Proceedings, Vol. 3144. CEUR-WS.org"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2021.3061664"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/S40037-022-00725-9"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/REW.2019.00051"},{"key":"e_1_3_2_13_2","unstructured":"Guru Bhandari and Gebremariam Assres. 2024. State-of-the-art and challenges of engineering ML-enabled software systems. Retrieved from https:\/\/github.com\/SmartSecLab\/ML-enabled-software-literature-review"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110542"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-56281-5_4"},{"key":"e_1_3_2_16_2","article-title":"Understanding the complexity and its impact on testing in ML-enabled systems","author":"Cao Junming","year":"2023","unstructured":"Junming Cao, Bihuan Chen, Longjie Hu, Jie Gao, Kaifeng Huang, and Xin Peng. 2023. Understanding the complexity and its impact on testing in ML-enabled systems. arXiv preprint arXiv:2301.03837 (2023).","journal-title":"arXiv preprint arXiv:2301.03837"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.clsr.2015.12.001"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2018.10.002"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3641289"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-30809-4_31"},{"key":"e_1_3_2_21_2","volume-title":"IEEE Standard Glossary of Software Engineering Terminology","author":"Committee IEEE Computer Society. Software Engineering Technical","year":"1983","unstructured":"IEEE Computer Society. Software Engineering Technical Committee. 1983. IEEE Standard Glossary of Software Engineering Terminology. Vol. 729. IEEE."},{"key":"e_1_3_2_22_2","volume-title":"Qualitative Inquiry and Research Design: Choosing among Five Approaches","author":"Creswell John W.","year":"2016","unstructured":"John W. Creswell and Cheryl N. Poth. 2016. Qualitative Inquiry and Research Design: Choosing among Five Approaches. Sage Publications."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/SCCC59417.2023.10315755"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion.2019.00024"},{"key":"e_1_3_2_25_2","article-title":"Classification, challenges, and automated approaches to handle non-functional requirements in ML-enabled systems: A systematic literature review","author":"Martino Vincenzo De","year":"2023","unstructured":"Vincenzo De Martino and Fabio Palomba. 2023. Classification, challenges, and automated approaches to handle non-functional requirements in ML-enabled systems: A systematic literature review. arXiv preprint arXiv:2311.17483 (2023).","journal-title":"arXiv preprint arXiv:2311.17483"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/FOSE.2007.20"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_2_28_2","doi-asserted-by":"crossref","unstructured":"Brenda Dervin. 1998. Sense-making theory and practice: An overview of user interests in knowledge seeking and use. Journal of knowledge Management 2 2 (1998) 36\u201346.","DOI":"10.1108\/13673279810249369"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","unstructured":"Sangeeta Dey and Seok-Won Lee. 2021. Multilayered review of safety approaches for machine learning-based systems in the days of AI. Journal of Systems & Software 176 (2021) 110941. DOI:10.1016\/j.jss.2021.110941","DOI":"10.1016\/j.jss.2021.110941"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.3390\/s22124637"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.5555\/3278358"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-97925-0_42"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-FoSE59343.2023.00008"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.3182\/20130522-3-BR-4036.00050"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1002\/9781119815075.ch9"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2017.4180835"},{"issue":"1","key":"e_1_3_2_37_2","first-page":"9","article-title":"A review of ChatGPT AI\u2019s impact on several business sectors","volume":"1","author":"George A. Shaji","year":"2023","unstructured":"A. Shaji George and A. S. Hovan George. 2023. A review of ChatGPT AI\u2019s impact on several business sectors. Partn. Univ. Int. Innov. J. 1, 1 (2023), 9\u201323.","journal-title":"Partn. Univ. Int. Innov. J."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.111031"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","unstructured":"Tuomas Granlund Vlad Stirbu and Tommi Mikkonen. 2021. Towards regulatory-compliant MLOps: Oravizio\u2019s journey from a machine learning experiment to a deployed certified medical product. SN Computer Science 2 5 (2021) 342. DOI:10.1007\/s42979-021-00726-1","DOI":"10.1007\/s42979-021-00726-1"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-21004-4_7"},{"issue":"1","key":"e_1_3_2_41_2","first-page":"1","article-title":"Data quality considerations for big data and machine learning: Going beyond data cleaning and transformations","volume":"10","author":"Gudivada Venkat","year":"2017","unstructured":"Venkat Gudivada, Amy Apon, and Junhua Ding. 2017. Data quality considerations for big data and machine learning: Going beyond data cleaning and transformations. Int. J. Advan. Softw. 10, 1 (2017), 1\u201320.","journal-title":"Int. J. Advan. Softw."},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3372787.3389300"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3104283"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","unstructured":"Khan Mohammad Habibullah Gregory Gay and Jennifer Horkoff. 2023. Non-functional requirements for machine learning: Understanding current use and challenges among practitioners. Requirements Engineering 28 2 (2023) 283\u2013316. DOI:10.1007\/s00766-022-00395-3","DOI":"10.1007\/s00766-022-00395-3"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSCON.2018.8369576"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-21388-5_48"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.5555\/2666527.2666528"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3546155.3546677"},{"key":"e_1_3_2_49_2","article-title":"Empirical analysis on CI\/CD pipeline evolution in machine learning projects","author":"Houerbi Alaa","year":"2024","unstructured":"Alaa Houerbi, Chadha Siala, Alexis Tucker, Dhia Elhaq Rzig, and Foyzul Hassan. 2024. Empirical analysis on CI\/CD pipeline evolution in machine learning projects. arXiv preprint arXiv:2403.12199 (2024).","journal-title":"arXiv preprint arXiv:2403.12199"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN-W54100.2022.00027"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/75110.75122"},{"key":"e_1_3_2_52_2","volume-title":"A Discipline for Software Engineering","author":"Humphrey Watts S.","year":"1995","unstructured":"Watts S. Humphrey. 1995. A Discipline for Software Engineering. Pearson Education India."},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.23919\/MIPRO.2018.8400261"},{"key":"e_1_3_2_54_2","unstructured":"IBM. 2023. What is software testing and how does it work? IBM. Retrieved from https:\/\/www.ibm.com\/topics\/software-testing"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3644815.3644976"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/CESSER-IP.2019.00009"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338955"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380378"},{"key":"e_1_3_2_59_2","volume-title":"CEUR Workshop Proceedings","volume":"3121","author":"J\u00fcngling Stephan","year":"2022","unstructured":"Stephan J\u00fcngling, Martin Peraic, and Cheng Zhu. 2022. Using the strategy design pattern for hybrid ai system design. In CEUR Workshop Proceedings, Vol. 3121. CEUR-WS.org."},{"key":"e_1_3_2_60_2","doi-asserted-by":"crossref","unstructured":"Barbara Kitchenham Pearl Brereton David Budgen Mark Turner John Bailey Stephen Linkman. 2009. Systematic literature reviews in software engineering\u2013a systematic literature review. Information and Software Technology 51 1 (2009) 7\u201315.","DOI":"10.1016\/j.infsof.2008.09.009"},{"key":"e_1_3_2_61_2","volume-title":"40th International Conference on Information Systems (ICIS\u201919","author":"Keller Robert","year":"2019","unstructured":"Robert Keller, Alexander Stohr, Gilbert Fridgen, Jannik Lockl, and Alexander Rieger. 2019. Affordance-experimentation-actualization theory in artificial intelligence research\u2014A predictive maintenance story. In 40th International Conference on Information Systems (ICIS\u201919)."},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2018.3571224"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3182515"},{"key":"e_1_3_2_64_2","doi-asserted-by":"crossref","unstructured":"Barbara Kitchenham and Pearl Brereton. 2013. A systematic review of systematic review process research in software engineering. Information and Software Tech. (2013) 2049\u20132075.","DOI":"10.1016\/j.infsof.2013.07.010"},{"key":"e_1_3_2_65_2","doi-asserted-by":"crossref","unstructured":"Barbara Kitchenham Rialette Pretorius David Budgen O. Pearl Brereton Mark Turner Mahmood Niazi and Stephen Linkman. 2010. Systematic literature reviews in software engineering\u2013a tertiary study. Information and Software Technology 52 8 (2010) 792\u2013805.","DOI":"10.1016\/j.infsof.2010.03.006"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-99229-7_36"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","unstructured":"Holger Klus Christoph Knieke Andreas Rausch and Stefan Wittek. 2023. Software engineering meets artificial intelligence. Electronic Communications of the EASST 82 (2023). DOI:10.14279\/tuj.eceasst.82.1224","DOI":"10.14279\/tuj.eceasst.82.1224"},{"key":"e_1_3_2_68_2","first-page":"2028","volume-title":"IEEE International Conference on Big Data (Big Data\u201920)","author":"Krasteva Iva","year":"2020","unstructured":"Iva Krasteva and Sylvia Ilieva. 2020. Adopting agile software development methodologies in big data projects\u2014A systematic literature review of experience reports. In IEEE International Conference on Big Data (Big Data\u201920). IEEE, 2028\u20132033."},{"key":"e_1_3_2_69_2","article-title":"ImageNet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. Advan. Neural Inf. Process. Syst. 25 (2012), 1097\u20131105.","journal-title":"Advan. Neural Inf. Process. Syst."},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.3233\/IDT-190160"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASEW.2019.00042"},{"issue":"1","key":"e_1_3_2_72_2","first-page":"101","article-title":"Applying TQM, CMM and ISO 9001 in knowledge management for software development process improvement","volume":"2","author":"Lee Ming-Chang","year":"2006","unstructured":"Ming-Chang Lee and To Chang. 2006. Applying TQM, CMM and ISO 9001 in knowledge management for software development process improvement. Int. J. Serv. Stand. 2, 1 (2006), 101\u2013115.","journal-title":"Int. J. Serv. Stand."},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-65854-0_4"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME52107.2021.00071"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3522664.3528607"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.23919\/SoftCOM50211.2020.9238323"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-19034-7_14"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3236082"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1145\/3487043"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2020.2984868"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/3183440.3183461"},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/RAISE.2019.00015"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/3486611.3492413"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER53432.2022.00155"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1109\/WAIN52551.2021.00026"},{"key":"e_1_3_2_86_2","unstructured":"Amna Pir Muhammad. 2021. Methods and guidelines for incorporating human factors requirements in automated vehicles development.CEUR Workshop Proceedings Vol. 2857. CEUR-WS.org."},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.nic.2020.07.004"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510209"},{"key":"e_1_3_2_89_2","article-title":"Software engineering for artificial intelligence and machine learning software: A systematic literature review","author":"Nascimento Elizamary","year":"2020","unstructured":"Elizamary Nascimento, Anh Nguyen-Duc, Ingrid Sundb\u00f8, and Tayana Conte. 2020. Software engineering for artificial intelligence and machine learning software: A systematic literature review. arXiv preprint arXiv:2011.03751 (2020).","journal-title":"arXiv preprint arXiv:2011.03751"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2023.111860"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1145\/1456536.1456597"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3417039"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.1954824"},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2020.2993662"},{"key":"e_1_3_2_95_2","doi-asserted-by":"publisher","DOI":"10.1145\/3522664.3528601"},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","unstructured":"Zhenpeng Chen Yanbin Cao Yuanqiang Liu Haoyu Wang Tao Xie and Xuanzhe Liu. 2020. A comprehensive study on challenges in deploying deep learning based software. In Proceedings of the 28th ACM Conference and Symposium on the Foundations of Software Engineering. 750\u2013762. DOI:10.1145\/3368089.3409759","DOI":"10.1145\/3368089.3409759"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533378"},{"key":"e_1_3_2_98_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2015.7332474"},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00130012"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.1109\/REW56159.2022.00039"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236112.3236130"},{"key":"e_1_3_2_102_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00090"},{"key":"e_1_3_2_103_2","doi-asserted-by":"publisher","unstructured":"Martin Rabe Stefan Milz and Patrick Mader. 2021. Development methodologies for safety critical machine learning applications in the automotive domain: A survey. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. 129\u2013141. DOI:10.1109\/CVPRW53098.2021.00023","DOI":"10.1109\/CVPRW53098.2021.00023"},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1145\/3526073.3527593"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-92966-6_6"},{"issue":"1","key":"e_1_3_2_106_2","first-page":"13","article-title":"A guide to writing the dissertation literature review","volume":"14","author":"Randolph Justus","year":"2009","unstructured":"Justus Randolph. 2009. A guide to writing the dissertation literature review. Pract. Assess., Res. Eval. 14, 1 (2009), 13.","journal-title":"Pract. Assess., Res. Eval."},{"key":"e_1_3_2_107_2","doi-asserted-by":"crossref","unstructured":"Gilberto Recupito Fabiano Pecorelli Gemma Catolino Valentina Lenarduzzi Davide Taibi Dario Di Nucci and Fabio Palomba. 2024. Technical debt in AI-enabled systems: On the prevalence severity impact and management strategies for code and architecture. Journal of Systems and Software 216 (2024) 112151.","DOI":"10.1016\/j.jss.2024.112151"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.1145\/234313.234423"},{"key":"e_1_3_2_109_2","volume-title":"Software Engineering: A Practitioner\u2019s Approach","author":"Roger S. Pressman","year":"2015","unstructured":"S. Pressman Roger and R. Maxin Bruce. 2015. Software Engineering: A Practitioner\u2019s Approach. McGraw-Hill Education."},{"key":"e_1_3_2_110_2","doi-asserted-by":"publisher","DOI":"10.1145\/1764810.1764814"},{"key":"e_1_3_2_111_2","volume-title":"Artificial Intelligence: A Modern Approach","author":"Russell Stuart J.","year":"2010","unstructured":"Stuart J. Russell and Peter Norvig. 2010. Artificial Intelligence: A Modern Approach. Pearson. London."},{"key":"e_1_3_2_112_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2022.107037"},{"key":"e_1_3_2_113_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigData47090.2019.9005591"},{"key":"e_1_3_2_114_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2021.3060483"},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICACCCN51052.2020.9362931"},{"key":"e_1_3_2_116_2","doi-asserted-by":"publisher","DOI":"10.1007\/s42979-021-00592-x"},{"key":"e_1_3_2_117_2","unstructured":"Sebastian Schelter Felix Biessmann Tim Januschowski David Salinas Stephan Seufert and Gyuri Szarvas. 2018. On challenges in machine learning model management. IEEE Data Engineering Bulletin 41 4 (2018) 5\u201315. http:\/\/sites.computer.org\/debull\/A18dec\/A18DEC-CD.pdf"},{"key":"e_1_3_2_118_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445538"},{"key":"e_1_3_2_119_2","article-title":"Hidden technical debt in machine learning systems","volume":"28","author":"Sculley David","year":"2015","unstructured":"David Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-Francois Crespo, and Dan Dennison. 2015. Hidden technical debt in machine learning systems. Advan. Neural Inf. Process. Syst. 28 (2015), 2503\u20132511.","journal-title":"Advan. Neural Inf. Process. Syst."},{"key":"e_1_3_2_120_2","article-title":"A large-scale study of model integration in ML-enabled software systems","author":"Sens Yorick","year":"2024","unstructured":"Yorick Sens, Henriette Knopp, Sven Peldszus, and Thorsten Berger. 2024. A large-scale study of model integration in ML-enabled software systems. arXiv preprint arXiv:2408.06226 (2024).","journal-title":"arXiv preprint arXiv:2408.06226"},{"key":"e_1_3_2_121_2","article-title":"A survey on sensor-based threats to internet-of-things (IoT) devices and applications","author":"Sikder Amit Kumar","year":"2018","unstructured":"Amit Kumar Sikder, Giuseppe Petracca, Hidayet Aksu, Trent Jaeger, and A. Selcuk Uluagac. 2018. A survey on sensor-based threats to internet-of-things (IoT) devices and applications. arXiv preprint arXiv:1802.02041 (2018).","journal-title":"arXiv preprint arXiv:1802.02041"},{"key":"e_1_3_2_122_2","doi-asserted-by":"publisher","DOI":"10.1109\/CSNT.2013.171"},{"key":"e_1_3_2_123_2","volume-title":"Software Engineering, 9\/E","author":"Sommerville Ian","year":"2011","unstructured":"Ian Sommerville. 2011. Software Engineering, 9\/E. Pearson Education India."},{"key":"e_1_3_2_124_2","doi-asserted-by":"publisher","DOI":"10.1109\/CCAA.2017.8229928"},{"key":"e_1_3_2_125_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2023.111615"},{"key":"e_1_3_2_126_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3517537"},{"key":"e_1_3_2_127_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIC50835.2020.9288582"},{"key":"e_1_3_2_128_2","doi-asserted-by":"publisher","DOI":"10.3233\/IDT-220252"},{"key":"e_1_3_2_129_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSECW53869.2021.00014"},{"key":"e_1_3_2_130_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2021.102402"},{"key":"e_1_3_2_131_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.comcom.2020.02.069"},{"issue":"3","key":"e_1_3_2_132_2","first-page":"40","article-title":"Analysis of requirement engineering processes, tools\/techniques and methodologies","volume":"5","author":"Rehman Tousif ur","year":"2013","unstructured":"Tousif ur Rehman, Muhammad Naeem Ahmed Khan, and Naveed Riaz. 2013. Analysis of requirement engineering processes, tools\/techniques and methodologies. Int. J. Inf. Technol. Comput. Sci. 5, 3 (2013), 40.","journal-title":"Int. J. Inf. Technol. Comput. Sci."},{"key":"e_1_3_2_133_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2022.3195432"},{"key":"e_1_3_2_134_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion52605.2021.00142"},{"key":"e_1_3_2_135_2","doi-asserted-by":"publisher","DOI":"10.1109\/HealthCom.2017.8210826"},{"key":"e_1_3_2_136_2","doi-asserted-by":"publisher","DOI":"10.1109\/JAS.2023.123486"},{"key":"e_1_3_2_137_2","doi-asserted-by":"crossref","unstructured":"Junjie Wang Yuchao Huang Chunyang Chen Zhe Liu Song Wang and Qing Wang. 2024. Software testing with large language models: Survey landscape and vision. IEEE Transactions on Software Engineering 50 4 (2024) 911\u2013936.","DOI":"10.1109\/TSE.2024.3368208"},{"key":"e_1_3_2_138_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3173346"},{"key":"e_1_3_2_139_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510071"},{"key":"e_1_3_2_140_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00034"},{"key":"e_1_3_2_141_2","doi-asserted-by":"publisher","DOI":"10.1109\/IWESEP49350.2019.00017"},{"key":"e_1_3_2_142_2","volume-title":"Sensemaking in Organizations","author":"Weick Karl E.","year":"1995","unstructured":"Karl E. Weick. 1995. Sensemaking in Organizations. Vol. 3. Sage Publications."},{"key":"e_1_3_2_143_2","doi-asserted-by":"publisher","DOI":"10.1109\/MAHC.2008.33"},{"key":"e_1_3_2_144_2","doi-asserted-by":"publisher","unstructured":"Christine T. Wolf and Drew Paine. 2020. Sensemaking practices in the everyday work of AI\/ML software engineeringIEEE\/ACM 42nd International Conference on Software Engineering Workshops (ICSEW\u201920). 86\u201392. DOI:10.1145\/3387940.3391496","DOI":"10.1145\/3387940.3391496"},{"key":"e_1_3_2_145_2","doi-asserted-by":"publisher","DOI":"10.1145\/3644815.3644951"},{"key":"e_1_3_2_146_2","article-title":"Large language models for cyber security: A systematic literature review","author":"Xu HanXiang","year":"2024","unstructured":"HanXiang Xu, ShenAo Wang, Ningke Li, Yanjie Zhao, Kai Chen, Kailong Wang, Yang Liu, Ting Yu, and HaoYu Wang. 2024. Large language models for cyber security: A systematic literature review. arXiv preprint arXiv:2405.04760 (2024).","journal-title":"arXiv preprint arXiv:2405.04760"},{"key":"e_1_3_2_147_2","doi-asserted-by":"publisher","DOI":"10.1109\/MNET.001.2100227"},{"key":"e_1_3_2_148_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC60848.2023.00022"},{"key":"e_1_3_2_149_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1023760326768"},{"key":"e_1_3_2_150_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00043"},{"key":"e_1_3_2_151_2","article-title":"A systematic literature review on large language models for automated program repair","author":"Zhang Quanjun","year":"2024","unstructured":"Quanjun Zhang, Chunrong Fang, Yang Xie, YuXiang Ma, Weisong Sun, and Yun Yang Zhenyu Chen. 2024. A systematic literature review on large language models for automated program repair. arXiv preprint arXiv:2405.01466 (2024).","journal-title":"arXiv preprint arXiv:2405.01466"},{"key":"e_1_3_2_152_2","article-title":"Software engineering practice in the development of deep learning applications","author":"Zhang Xufan","year":"2019","unstructured":"Xufan Zhang, Yilin Yang, Yang Feng, and Zhenyu Chen. 2019. Software engineering practice in the development of deep learning applications. arXiv preprint arXiv:1910.03156 (2019).","journal-title":"arXiv preprint arXiv:1910.03156"},{"key":"e_1_3_2_153_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00043"},{"key":"e_1_3_2_154_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-56281-5_7"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3731597","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3731597","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:57:19Z","timestamp":1750298239000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3731597"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,6]]},"references-count":153,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2025,10,31]]}},"alternative-id":["10.1145\/3731597"],"URL":"https:\/\/doi.org\/10.1145\/3731597","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,6]]},"assertion":[{"value":"2023-11-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-06","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}