{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T12:02:05Z","timestamp":1774958525675,"version":"3.50.1"},"publisher-location":"Cham","reference-count":25,"publisher":"Springer International Publishing","isbn-type":[{"value":"9783319916019","type":"print"},{"value":"9783319916026","type":"electronic"}],"license":[{"start":{"date-parts":[[2018,1,1]],"date-time":"2018-01-01T00:00:00Z","timestamp":1514764800000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p><jats:bold>Context<\/jats:bold>: Continuous experimentation is frequently used in web-facing companies and it is starting to gain the attention of embedded systems companies. However, embedded systems companies have different challenges and requirements to run experiments in their systems. <jats:bold>Objective<\/jats:bold>: This paper explores the challenges during the adoption of continuous experimentation in embedded systems from both industry practice and academic research. It presents strategies, guidelines, and solutions to overcome each of the identified challenges. <jats:bold>Method<\/jats:bold>: This research was conducted in two parts. The first part is a literature review with the aim to analyze the challenges in adopting continuous experimentation from the research perspective. The second part is a multiple case study based on interviews and workshop sessions with five companies to understand the challenges from the industry perspective and how they are working to overcome them. <jats:bold>Results<\/jats:bold>: This study found a set of twelve challenges divided into three areas; technical, business, and organizational challenges and strategies grouped into three categories, architecture, data handling and development processes. <jats:bold>Conclusions<\/jats:bold>: The set of identified challenges are presented with a set of strategies, guidelines, and solutions. To the knowledge of the authors, this paper is the first to provide an extensive list of challenges and strategies for continuous experimentation in embedded systems. Moreover, this research points out open challenges and the need for new tools and novel solutions for the further development of experimentation in embedded systems.<\/jats:p>","DOI":"10.1007\/978-3-319-91602-6_20","type":"book-chapter","created":{"date-parts":[[2018,5,16]],"date-time":"2018-05-16T13:13:43Z","timestamp":1526476423000},"page":"277-292","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":40,"title":["Challenges and Strategies for Undertaking Continuous Experimentation to Embedded Systems: Industry and Research Perspectives"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2501-9926","authenticated-orcid":false,"given":"David Issa","family":"Mattos","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2854-722X","authenticated-orcid":false,"given":"Jan","family":"Bosch","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7700-1816","authenticated-orcid":false,"given":"Helena Holmstr\u00f6m","family":"Olsson","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2018,5,17]]},"reference":[{"key":"20_CR1","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1016\/j.infsof.2016.04.008","volume":"77","author":"E Lindgren","year":"2016","unstructured":"Lindgren, E., M\u00fcnch, J.: Raising the odds of success: the current state of experimentation in product development. Inf. Softw. Technol. 77, 80\u201391 (2016)","journal-title":"Inf. Softw. Technol."},{"key":"20_CR2","doi-asserted-by":"crossref","unstructured":"Eliasson, U., Heldal, R., Knauss, E., Pelliccione, P.: The need of complementing plan-driven requirements engineering with emerging communication: experiences from Volvo Car Group. In: Proceedings of 2015 IEEE 23rd International Requirements Engineering Conference RE 2015, pp. 372\u2013381 (2015)","DOI":"10.1109\/RE.2015.7320454"},{"key":"20_CR3","doi-asserted-by":"crossref","unstructured":"Olsson, H.H., Bosch, J.: From opinions to data-driven software R&D: a multi-case study on how to close the \u2018open loop\u2019 problem. In: Proceedings of 40th Euromicro Conference Series on Software Engineering and Advanced Applications SEAA 2014, pp. 9\u201316 (2014)","DOI":"10.1109\/SEAA.2014.75"},{"key":"20_CR4","doi-asserted-by":"crossref","unstructured":"Fabijan, A., Dmitriev, P., Olsson, H.H., Bosch, J.: The evolution of continuous experimentation in software product development. In: Proceedings of the 39th International Conference on Software Engineering ICSE 2017 (2017)","DOI":"10.1109\/ICSE.2017.76"},{"key":"20_CR5","doi-asserted-by":"crossref","unstructured":"Tang, D., Agarwal, A., O\u2019Brien, D., Meyer, M.: Overlapping experiment infrastructure. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining-KDD 2010, p. 17 (2010)","DOI":"10.1145\/1835804.1835810"},{"issue":"1","key":"20_CR6","doi-asserted-by":"publisher","first-page":"140","DOI":"10.1007\/s10618-008-0114-1","volume":"18","author":"R Kohavi","year":"2009","unstructured":"Kohavi, R., Longbotham, R., Sommerfield, D., Henne, R.M.: Controlled experiments on the web: survey and practical guide. Data Min. Knowl. Discov. 18(1), 140\u2013181 (2009)","journal-title":"Data Min. Knowl. Discov."},{"key":"20_CR7","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1007\/978-3-319-11283-1_2","volume-title":"Continuous Software Engineering","author":"HH Olsson","year":"2014","unstructured":"Olsson, H.H., Bosch, J.: Climbing the \u201cStairway to Heaven\u201d: evolving from agile development to continuous deployment of software. In: Bosch, J. (ed.) Continuous Software Engineering, pp. 15\u201327. Springer, Cham (2014). https:\/\/doi.org\/10.1007\/978-3-319-11283-1_2"},{"key":"20_CR8","series-title":"Lecture Notes in Business Information Processing","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1007\/978-3-642-30746-1_3","volume-title":"Software Business","author":"J Bosch","year":"2012","unstructured":"Bosch, J.: Building products as innovation experiment systems. In: Cusumano, M.A., Iyer, B., Venkatraman, N. (eds.) ICSOB 2012. LNBIP, vol. 114, pp. 27\u201339. Springer, Heidelberg (2012). https:\/\/doi.org\/10.1007\/978-3-642-30746-1_3"},{"key":"20_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/978-3-319-11283-1_13","volume-title":"Continuous Software Engineering","author":"HH Olsson","year":"2014","unstructured":"Olsson, H.H., Bosch, J.: The HYPEX model: from opinions to data-driven software development. In: Bosch, J. (ed.) Continuous Software Engineering, pp. 1\u2013226. Springer, Cham (2014). https:\/\/doi.org\/10.1007\/978-3-319-11283-1_13"},{"key":"20_CR10","series-title":"Lecture Notes in Business Information Processing","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1007\/978-3-319-33515-5_4","volume-title":"Agile Processes, in Software Engineering, and Extreme Programming","author":"A Fabijan","year":"2016","unstructured":"Fabijan, A., Olsson, H.H., Bosch, J.: The lack of sharing of customer data in large software organizations: challenges and implications. In: Sharp, H., Hall, T. (eds.) XP 2016. LNBIP, vol. 251, pp. 39\u201352. Springer, Cham (2016). https:\/\/doi.org\/10.1007\/978-3-319-33515-5_4"},{"key":"20_CR11","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1016\/j.jss.2016.03.034","volume":"123","author":"F Fagerholm","year":"2017","unstructured":"Fagerholm, F., Sanchez Guinea, A., M\u00e4enp\u00e4\u00e4, H., M\u00fcnch, J.: The RIGHT model for continuous experimentation. J. Syst. Softw. 123, 292\u2013305 (2017)","journal-title":"J. Syst. Softw."},{"key":"20_CR12","volume-title":"The Lean Startup: How Today\u2019s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses","author":"E Ries","year":"2011","unstructured":"Ries, E.: The Lean Startup: How Today\u2019s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses, 1st edn. Crown Publishing Group, New York (2011)","edition":"1"},{"key":"20_CR13","doi-asserted-by":"crossref","unstructured":"Kohavi, R., Deng, A., Longbotham, R., Xu, Y.: Seven rules of thumb for web site experimenters. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining-KDD 2014, pp. 1857\u20131866 (2014)","DOI":"10.1145\/2623330.2623341"},{"key":"20_CR14","unstructured":"Olsson, H.H., Bosch, J.: So much data\u202f; so little value\u202f: a multi-case study on improving the impact of data-driven development practices. In: Proceedings of the Ibero American Conference on Software Engineering (ClbSE), 22nd\u201323rd May, Buenos Aires, Argentina (2017)"},{"key":"20_CR15","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1007\/978-3-642-34026-0_3","volume-title":"Leveraging Applications of Formal Methods, Verification and Validation. Technologies for Mastering Change","author":"J Bosch","year":"2012","unstructured":"Bosch, J., Eklund, U.: Eternal embedded software: towards innovation experiment systems. In: Margaria, T., Steffen, B. (eds.) ISoLA 2012. LNCS, vol. 7609, pp. 19\u201331. Springer, Heidelberg (2012). https:\/\/doi.org\/10.1007\/978-3-642-34026-0_3"},{"key":"20_CR16","doi-asserted-by":"crossref","unstructured":"Giaimo, F., Berger, C.: Design criteria to architect continuous experimentation for self-driving vehicles. In: 2017 IEEE International Conference on Software Architecture (ICSA), pp. 203\u2013210 (2017)","DOI":"10.1109\/ICSA.2017.36"},{"key":"20_CR17","doi-asserted-by":"crossref","unstructured":"Bosch, J., Olsson, H.H.: Data-driven continuous evolution of smart systems. In: Proceedings of the 11th International Workshop on Software Engineering for Adaptive and Self-Managing Systems-SEAMS 2016, pp. 28\u201334 (2016)","DOI":"10.1145\/2897053.2897066"},{"key":"20_CR18","doi-asserted-by":"crossref","unstructured":"Mattos, D.I., Bosch, J., Olsson, H.H.: Your system gets better every day you use it: towards automated continuous experimentation. In: Proceedings of the 43th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) (2017)","DOI":"10.1109\/SEAA.2017.15"},{"key":"20_CR19","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"publisher","first-page":"146","DOI":"10.1007\/978-3-319-69926-4_12","volume-title":"Product-Focused Software Process Improvement","author":"DI Mattos","year":"2017","unstructured":"Mattos, D.I., Bosch, J., Holmstr\u00f6m Olsson, H.: More for less: automated experimentation in software-intensive systems. In: Felderer, M., M\u00e9ndez Fern\u00e1ndez, D., Turhan, B., Kalinowski, M., Sarro, F., Winkler, D. (eds.) PROFES 2017. LNCS, vol. 10611, pp. 146\u2013161. Springer, Cham (2017). https:\/\/doi.org\/10.1007\/978-3-319-69926-4_12"},{"key":"20_CR20","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-29044-2","volume-title":"Experimentation in Software Engineering","author":"Claes Wohlin","year":"2012","unstructured":"Wohlin, C., Runeson, P., H\u00f6st, M., Ohlsson, M.C., Regnell, B., Wessl\u00e9n, A.: Experimentation in Software Engineering, vol. 1. Springer, Heidelberg (2012). https:\/\/doi.org\/10.1007\/978-3-642-29044-2"},{"issue":"2","key":"20_CR21","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1007\/s10664-008-9102-8","volume":"14","author":"P Runeson","year":"2009","unstructured":"Runeson, P., H\u00f6st, M.: Guidelines for conducting and reporting case study research in software engineering. Empir. Softw. Eng. 14(2), 131\u2013164 (2009)","journal-title":"Empir. Softw. Eng."},{"key":"20_CR22","unstructured":"Optimizely, \u201cOptimizely.\u201d https:\/\/www.optimizely.com\/. Accessed 28 June 2017"},{"key":"20_CR23","unstructured":"Zhang, B.: Privacy Concerns in Online Recommender Systems: Influences of Control and User Data Input, pp. 159\u2013173 (2014)"},{"key":"20_CR24","doi-asserted-by":"publisher","first-page":"e1876","DOI":"10.1002\/smr.1876","volume":"29","author":"H Holmstr\u00f6m Olsson","year":"2017","unstructured":"Holmstr\u00f6m Olsson, H., Bosch, J.: From ad hoc to strategic ecosystem management: the Three-Layer Ecosystem Strategy Model? (TeLESM). J. Softw. Evol. Process 29, e1876 (2017)","journal-title":"J. Softw. Evol. Process"},{"key":"20_CR25","unstructured":"Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Dennison, D.: Hidden Technical debt in machine learning systems. In: NIPS, pp. 2494\u20132502 (2015)"}],"container-title":["Lecture Notes in Business Information Processing","Agile Processes in Software Engineering and Extreme Programming"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-319-91602-6_20","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,13]],"date-time":"2024-03-13T18:48:24Z","timestamp":1710355704000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-319-91602-6_20"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018]]},"ISBN":["9783319916019","9783319916026"],"references-count":25,"URL":"https:\/\/doi.org\/10.1007\/978-3-319-91602-6_20","relation":{},"ISSN":["1865-1348","1865-1356"],"issn-type":[{"value":"1865-1348","type":"print"},{"value":"1865-1356","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018]]},"assertion":[{"value":"17 May 2018","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"XP","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"International Conference on Agile Software Development","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Porto","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Portugal","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2018","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"21 May 2018","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"25 May 2018","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"19","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"xpu2018","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/www.agilealliance.org\/xp2018\/","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}}]}}