{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,28]],"date-time":"2025-09-28T04:19:00Z","timestamp":1759033140719,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":19,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,11,8]],"date-time":"2020-11-08T00:00:00Z","timestamp":1604793600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,11,8]]},"DOI":"10.1145\/3368089.3417039","type":"proceedings-article","created":{"date-parts":[[2020,11,10]],"date-time":"2020-11-10T21:08:44Z","timestamp":1605042524000},"page":"1513-1516","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Continuous experimentation on artificial intelligence software: a research agenda"],"prefix":"10.1145","author":[{"given":"Anh","family":"Nguyen-Duc","sequence":"first","affiliation":[{"name":"University of South Eastern Norway, Norway"}]},{"given":"Pekka","family":"Abrahamsson","sequence":"additional","affiliation":[{"name":"University of Jyv\u00e4skyl\u00e4, Finland"}]}],"member":"320","published-online":{"date-parts":[[2020,11,8]]},"reference":[{"key":"e_1_3_2_2_1_1","volume-title":"Characterizing machine learning process: A maturity framework. ([n. d.]). arXiv","author":"Akkiraju Rama","year":"1811","unstructured":"Rama Akkiraju , Vibha Sinha , Anbang Xu , Jalal Mahmud , Pritam Gundecha , Zhe Liu , Xiaotong Liu , and John Schumacher . [n.d.]. Characterizing machine learning process: A maturity framework. ([n. d.]). arXiv : 1811 .04871 http:\/\/arxiv.org\/abs\/ 1811.04871 Rama Akkiraju, Vibha Sinha, Anbang Xu, Jalal Mahmud, Pritam Gundecha, Zhe Liu, Xiaotong Liu, and John Schumacher. [n.d.]. Characterizing machine learning process: A maturity framework. ([n. d.]). arXiv: 1811.04871 http:\/\/arxiv.org\/abs\/ 1811.04871"},{"key":"e_1_3_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSESEIP.2019.00042"},{"key":"e_1_3_2_2_3_1","volume-title":"Engineering AI Systems: A Research Agenda. ([n. d.]). arXiv","author":"Bosch Jan","year":"2001","unstructured":"Jan Bosch , Ivica Crnkovic , and Helena Holmstr\u00f6m Olsson . [n.d.]. Engineering AI Systems: A Research Agenda. ([n. d.]). arXiv : 2001 .07522 http:\/\/arxiv.org\/abs\/ 2001.07522 Jan Bosch, Ivica Crnkovic, and Helena Holmstr\u00f6m Olsson. [n.d.]. Engineering AI Systems: A Research Agenda. ([n. d.]). arXiv: 2001.07522 http:\/\/arxiv.org\/abs\/ 2001.07522"},{"volume-title":"Agile Processes, in Software Engineering, and Extreme Programming ( 2016 ) (Lecture Notes in Business Information Processing)","author":"Duc Anh Nguyen","key":"e_1_3_2_2_4_1","unstructured":"Anh Nguyen Duc and Pekka Abrahamsson . [n.d.]. Minimum Viable Product or Multiple Facet Product? The Role of MVP in Software Startups . In Agile Processes, in Software Engineering, and Extreme Programming ( 2016 ) (Lecture Notes in Business Information Processing) , Helen Sharp and Tracy Hall (Eds.). Springer International Publishing , 118-130. Anh Nguyen Duc and Pekka Abrahamsson. [n.d.]. Minimum Viable Product or Multiple Facet Product? The Role of MVP in Software Startups. In Agile Processes, in Software Engineering, and Extreme Programming ( 2016 ) (Lecture Notes in Business Information Processing), Helen Sharp and Tracy Hall (Eds.). Springer International Publishing, 118-130."},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2016.03.034"},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2015.06.063"},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2019.00050"},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884783"},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA.2011.62"},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-19034-7_14"},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2019.2954841"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3383219.3383220"},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA.2012.54"},{"key":"e_1_3_2_2_14_1","volume-title":"Continuous Integration of Machine Learning Models with ease. ml\/ci: Towards a Rigorous Yet Practical Treatment. ([n. d.]). arXiv","author":"Renggli Cedric","year":"1903","unstructured":"Cedric Renggli , Bojan Karla\u0161 , Bolin Ding , Feng Liu , Kevin Schawinski , Wentao Wu , and Ce Zhang . [n.d.]. Continuous Integration of Machine Learning Models with ease. ml\/ci: Towards a Rigorous Yet Practical Treatment. ([n. d.]). arXiv : 1903 .00278 http:\/\/arxiv.org\/abs\/ 1903.00278 Cedric Renggli, Bojan Karla\u0161, Bolin Ding, Feng Liu, Kevin Schawinski, Wentao Wu, and Ce Zhang. [n.d.]. Continuous Integration of Machine Learning Models with ease. ml\/ci: Towards a Rigorous Yet Practical Treatment. ([n. d.]). arXiv: 1903.00278 http:\/\/arxiv.org\/abs\/ 1903.00278"},{"key":"e_1_3_2_2_15_1","unstructured":"Eric Ries. [n.d.]. The lean startup: how today's entrepreneurs use continuous innovation to create radically successful businesses. OCLC: 693809631.  Eric Ries. [n.d.]. The lean startup: how today's entrepreneurs use continuous innovation to create radically successful businesses. OCLC: 693809631."},{"key":"e_1_3_2_2_16_1","first-page":"2503","volume-title":"Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 2 ( 2015-12-07)","author":"Sculley D.","unstructured":"D. Sculley , Gary Holt , Daniel Golovin , Eugene Davydov , Todd Phillips , Dietmar Ebner , Vinay Chaudhary , Michael Young , Jean-Francois Crespo , and Dan Dennison . [n.d.]. Hidden technical debt in Machine learning systems . In Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 2 ( 2015-12-07) (NIPS'15). MIT Press , 2503 - 2511 . D. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-Francois Crespo, and Dan Dennison. [n.d.]. Hidden technical debt in Machine learning systems. In Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 2 ( 2015-12-07) (NIPS'15). MIT Press, 2503-2511."},{"key":"e_1_3_2_2_17_1","volume-title":"ECCOLA-a Method for Implementing Ethically Aligned AI Systems. ([n. d.]). arXiv","author":"Vakkuri Ville","year":"2004","unstructured":"Ville Vakkuri , Kai-Kristian Kemell , and Pekka Abrahamsson . [n.d.]. ECCOLA-a Method for Implementing Ethically Aligned AI Systems. ([n. d.]). arXiv : 2004 .08377 http:\/\/arxiv.org\/abs\/ 2004.08377 Ville Vakkuri, Kai-Kristian Kemell, and Pekka Abrahamsson. [n.d.]. ECCOLA-a Method for Implementing Ethically Aligned AI Systems. ([n. d.]). arXiv: 2004.08377 http:\/\/arxiv.org\/abs\/ 2004.08377"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3076246.3076248"},{"key":"e_1_3_2_2_19_1","volume-title":"Machine Learning Testing: Survey, Landscapes and Horizons. ([n. d.]). arXiv","author":"Zhang Jie M.","year":"1906","unstructured":"Jie M. Zhang , Mark Harman , Lei Ma , and Yang Liu . [n.d.]. Machine Learning Testing: Survey, Landscapes and Horizons. ([n. d.]). arXiv : 1906 .10742 http:\/\/arxiv.org\/abs\/ 1906.10742 Jie M. Zhang, Mark Harman, Lei Ma, and Yang Liu. [n.d.]. Machine Learning Testing: Survey, Landscapes and Horizons. ([n. d.]). arXiv: 1906.10742 http:\/\/arxiv.org\/abs\/ 1906.10742"}],"event":{"name":"ESEC\/FSE '20: 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering"],"location":"Virtual Event USA","acronym":"ESEC\/FSE '20"},"container-title":["Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3368089.3417039","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3368089.3417039","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:01:58Z","timestamp":1750197718000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3368089.3417039"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,8]]},"references-count":19,"alternative-id":["10.1145\/3368089.3417039","10.1145\/3368089"],"URL":"https:\/\/doi.org\/10.1145\/3368089.3417039","relation":{},"subject":[],"published":{"date-parts":[[2020,11,8]]},"assertion":[{"value":"2020-11-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}