{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T07:00:19Z","timestamp":1781593219975,"version":"3.54.5"},"reference-count":34,"publisher":"SAGE Publications","issue":"4","license":[{"start":{"date-parts":[[2020,2,10]],"date-time":"2020-02-10T00:00:00Z","timestamp":1581292800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IDT"],"published-print":{"date-parts":[[2020,2,10]]},"abstract":"<jats:p>Machine learning techniques, especially deep learning, have achieved remarkable breakthroughs over the past decade. At present, machine learning applications are deployed in many fields. However, the outcomes of software engineering researches are not always easily utilized in the development and deployment of machine learning applications. The main reason for this difficulty is the many differences between machine learning applications and traditional information systems. Machine learning techniques are evolving rapidly, but face inherent technical and non-technical challenges that complicate their lifecycle activities. This review paper attempts to clarify the software engineering challenges for machine learning applications that either exist or potentially exist by conducting a systematic literature collection and by mapping the identified challenge topics to knowledge areas defined by the Software Engineering Body of Knowledge (Swebok).<\/jats:p>","DOI":"10.3233\/idt-190160","type":"journal-article","created":{"date-parts":[[2020,2,11]],"date-time":"2020-02-11T10:46:07Z","timestamp":1581417967000},"page":"463-476","source":"Crossref","is-referenced-by-count":63,"title":["Software engineering challenges for machine learning applications: A literature review"],"prefix":"10.1177","volume":"13","author":[{"given":"Fumihiro","family":"Kumeno","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","reference":[{"key":"10.3233\/IDT-190160_ref9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.infsof.2015.03.007","article-title":"Guidelines for conducting systematic mapping studies in software engineering: An update","volume":"64","author":"Petersen","year":"2015","journal-title":"Information and Software Technology"},{"issue":"2","key":"10.3233\/IDT-190160_ref26","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1145\/3299887.3299891","article-title":"Data lifecycle challenges in production machine learning: A survey","volume":"47","author":"Polyzotis","year":"2018","journal-title":"SIGMOD Rec"},{"issue":"4","key":"10.3233\/IDT-190160_ref27","first-page":"5","article-title":"On challenges in machine learning model management","volume":"41","author":"Sebastian","year":"2018","journal-title":"IEEE Data Eng. Bull"},{"key":"10.3233\/IDT-190160_ref31","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.sigpro.2013.12.026","article-title":"Review: A review of novelty detection","volume":"99","author":"Pimentel","year":"2014","journal-title":"Signal Process"},{"issue":"6","key":"10.3233\/IDT-190160_ref32","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1145\/3282486","article-title":"The challenge of crafting intelligible intelligence","volume":"62","author":"Weld","year":"2019","journal-title":"Commun. ACM"},{"issue":"1","key":"10.3233\/IDT-190160_ref33","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1080\/21693277.2016.1192517","article-title":"Machine learning in manufacturing: Advantages, challenges, and applications","volume":"4","author":"Wuest","year":"2016","journal-title":"Production & Manufacturing Research"},{"issue":"11","key":"10.3233\/IDT-190160_ref39","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1109\/TSE.2017.2754374","article-title":"Data scientists in software teams: State of the art and challenges","volume":"44","author":"Kim","year":"2017","journal-title":"IEEE Transactions on Software Engineering"},{"issue":"2","key":"10.3233\/IDT-190160_ref43","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1145\/1964897.1964906","article-title":"Inactive learning?: Difficulties employing active learning in practice","volume":"12","author":"Attenberg","year":"2011","journal-title":"SIGKDD Explor. Newsl"},{"key":"10.3233\/IDT-190160_ref44","doi-asserted-by":"crossref","first-page":"514","DOI":"10.1109\/ACCESS.2014.2325029","article-title":"Big data deep learning: Challenges and perspectives","volume":"2","author":"Chen","year":"2014","journal-title":"in IEEE Access"},{"issue":"4","key":"10.3233\/IDT-190160_ref49","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1609\/aimag.v36i4.2577","article-title":"Research priorities for robust and beneficial artificial intelligence","volume":"36","author":"Russell","year":"2015","journal-title":"Ai Magazine"},{"key":"10.3233\/IDT-190160_ref56","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1016\/j.cag.2018.09.018","article-title":"A task-and-technique centered survey on visual analytics for deep learning model engineering","volume":"77","author":"Garcia","year":"2018","journal-title":"Computers & Graphics"},{"issue":"1","key":"10.3233\/IDT-190160_ref62","first-page":"2859","article-title":"Linear dimensionality reduction: Survey, insights, and generalizations","volume":"16","author":"Cunningham","year":"2015","journal-title":"The Journal of Machine Learning Research"},{"issue":"3","key":"10.3233\/IDT-190160_ref63","doi-asserted-by":"crossref","first-page":"1249","DOI":"10.1109\/TVCG.2016.2640960","article-title":"Visualizing high-dimensional data: advances in the past decade","volume":"23","author":"Liu","year":"2017","journal-title":"in IEEE Transactions on Visualization and Computer Graphics"},{"key":"10.3233\/IDT-190160_ref68","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1214\/09-SS054","article-title":"A survey of cross-validation procedures for model selection","volume":"4","author":"Arlot","year":"2010","journal-title":"Statistics Surveys"},{"key":"10.3233\/IDT-190160_ref69","doi-asserted-by":"publisher","DOI":"10.1145\/2523813"},{"issue":"6","key":"10.3233\/IDT-190160_ref70","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1145\/3136625","article-title":"Feature selection: A data perspective","volume":"50","author":"Li","year":"2018","journal-title":"ACM Computing Surveys (CSUR)"},{"issue":"1","key":"10.3233\/IDT-190160_ref71","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1609\/aimag.v33i1.2367","article-title":"Challenges and opportunities in applied machine learning","volume":"33","author":"Brodley","year":"2012","journal-title":"Ai Magazine"},{"key":"10.3233\/IDT-190160_ref72","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13174-018-0087-2","article-title":"A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities","volume":"9","author":"Boutaba","year":"2018","journal-title":"Journal of Internet Services and Applications"},{"key":"10.3233\/IDT-190160_ref75","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1016\/j.neucom.2017.01.078","article-title":"A survey on data preprocessing for data stream mining","volume":"239","author":"Ramrez-Gallego","year":"2017","journal-title":"Neurocomput"},{"key":"10.3233\/IDT-190160_ref79","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1007\/s13748-019-00185-z","article-title":"A review on the self and dual interactions between machine learning and optimization","volume":"8","author":"Heda","year":"2019","journal-title":"Progress in Artificial Intelligence"},{"key":"10.3233\/IDT-190160_ref86","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.neucom.2016.12.038","article-title":"A survey of deep neural network architectures and their applications","volume":"234","author":"Liu","year":"2017","journal-title":"Neurocomputing"},{"issue":"1","key":"10.3233\/IDT-190160_ref89","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1007\/s00521-006-0039-9","article-title":"Developing artificial neural networks for safety critical systems","volume":"16","author":"Kurd","year":"2007","journal-title":"Neural Computing & Applications"},{"issue":"1","key":"10.3233\/IDT-190160_ref90","doi-asserted-by":"crossref","first-page":"15","DOI":"10.4271\/2016-01-0128","article-title":"Challenges in autonomous vehicle testing and validation","volume":"4","author":"Koopman","year":"2016","journal-title":"SAE Int. J. Trans. Safety"},{"issue":"8","key":"10.3233\/IDT-190160_ref94","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1016\/j.ijhcs.2009.03.004","article-title":"Interacting meaningfully with machine learning systems: Three experiments","volume":"67","author":"Stumpf","year":"2009","journal-title":"Int. J. Hum.-Comput. Stud"},{"issue":"4","key":"10.3233\/IDT-190160_ref96","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1609\/aimag.v35i4.2513","article-title":"Power to the people: The role of humans in interactive machine learning","volume":"35","author":"Amershi","year":"2014","journal-title":"AI Magazine"},{"issue":"4","key":"10.3233\/IDT-190160_ref99","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1007\/s10339-011-0404-1","article-title":"Model learning for robot control: A survey","volume":"12","author":"Nguyen-Tuong","year":"2011","journal-title":"Cognitive Processing"},{"issue":"5","key":"10.3233\/IDT-190160_ref100","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1016\/j.robot.2008.10.024","article-title":"A survey of robot learning from demonstration","volume":"57","author":"Argall","year":"2009","journal-title":"Robotics and Autonomous Systems"},{"issue":"3","key":"10.3233\/IDT-190160_ref103","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1089\/big.2016.0051","article-title":"On the safety of machine learning: Cyber-physical systems, decision sciences, and data products","volume":"5","author":"Varshney","year":"2017","journal-title":"Big Data"},{"issue":"3","key":"10.3233\/IDT-190160_ref105","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1609\/aimag.v38i3.2741","article-title":"European Union regulations on algorithmic decision-making and a right to explanation","volume":"38","author":"Goodman","year":"2017","journal-title":"Ai Magazine"},{"key":"10.3233\/IDT-190160_ref106","first-page":"334","article-title":"The ethics of artificial intelligence","volume":"316","author":"Bostrom","year":"2014","journal-title":"The Cambridge Handbook of Artificial Intelligence"},{"key":"10.3233\/IDT-190160_ref111","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1007\/s10462-013-9406-y","article-title":"Metalearning: A survey of trends and technologies","volume":"44","author":"Lemke","year":"2015","journal-title":"Artificial Intelligence Review"},{"key":"10.3233\/IDT-190160_ref113","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s13721-016-0125-6","article-title":"A review of automatic selection methods for machine learning algorithms and hyper-parameter values","volume":"5","author":"Luo","year":"2016","journal-title":"Network Modeling Analysis in Health Informatics and Bioinformatics"},{"key":"10.3233\/IDT-190160_ref116","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-016-0043-6"},{"issue":"4","key":"10.3233\/IDT-190160_ref121","first-page":"e1253","article-title":"Deep learning for sentiment analysis: A survey","volume":"8","author":"Zhang","year":"2018","journal-title":"Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery"}],"container-title":["Intelligent Decision Technologies"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/IDT-190160","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:22:38Z","timestamp":1777454558000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/IDT-190160"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,10]]},"references-count":34,"journal-issue":{"issue":"4"},"URL":"https:\/\/doi.org\/10.3233\/idt-190160","relation":{},"ISSN":["1872-4981","1875-8843"],"issn-type":[{"value":"1872-4981","type":"print"},{"value":"1875-8843","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,10]]}}}