{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T02:26:17Z","timestamp":1775874377014,"version":"3.50.1"},"reference-count":44,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,10,17]],"date-time":"2022-10-17T00:00:00Z","timestamp":1665964800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,17]],"date-time":"2022-10-17T00:00:00Z","timestamp":1665964800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"University of Helsinki including Helsinki University Central Hospital"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Software Qual J"],"published-print":{"date-parts":[[2023,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Continuous software engineering has become commonplace in numerous fields. However, in regulating intensive sectors, where additional concerns need to be taken into account, it is often considered difficult to apply continuous development approaches, such as devops. In this paper, we present an approach for using pull requests as design controls, and apply this approach to machine learning in certified medical systems leveraging model cards, a novel technique developed to add explainability to machine learning systems, as a regulatory audit trail. The approach is demonstrated with an industrial system that we have used previously to show how medical systems can be developed in a continuous fashion.<\/jats:p>","DOI":"10.1007\/s11219-022-09601-5","type":"journal-article","created":{"date-parts":[[2022,10,17]],"date-time":"2022-10-17T13:09:27Z","timestamp":1666012167000},"page":"307-333","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Continuous design control for machine learning in certified medical systems"],"prefix":"10.1007","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9462-5922","authenticated-orcid":false,"given":"Vlad","family":"Stirbu","sequence":"first","affiliation":[]},{"given":"Tuomas","family":"Granlund","sequence":"additional","affiliation":[]},{"given":"Tommi","family":"Mikkonen","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,10,17]]},"reference":[{"key":"9601_CR1","doi-asserted-by":"crossref","unstructured":"Aho, T., Sievi-Korte, O., Kilamo, T., Yaman, S., Mikkonen, T. (2020). Demystifying data science projects: A look on the people and process of data science today. In: International Conference on Product-focused Software Process Improvement (PROFES\u201920), pp. 153\u2013167. Springer.","DOI":"10.1007\/978-3-030-64148-1_10"},{"key":"9601_CR2","unstructured":"AWS Solutions. (2021). AWS MLOps Framework. https:\/\/docs.aws.amazon.com\/solutions\/ latest\/aws-mlops-framework\/welcome.html. Retrieved 14 March\u00a02021."},{"key":"9601_CR3","unstructured":"Bass, L., Weber, I., Zhu, L. (2015). DevOps: A Software Architect\u2019s Perspective. Addison-Wesley Professional."},{"key":"9601_CR4","doi-asserted-by":"publisher","unstructured":"Baylor, D., Breck, E., Cheng, H.-T., Fiedel, N., Foo, C. Y., Haque, Z., Haykal, S., Ispir, M., Jain, V., Koc, L., Koo, C. Y., Lew, L., Mewald, C., Modi, A. N., Polyzotis, N., Ramesh, S., Roy, S., Whang, S. E., Wicke, M., Wilkiewicz, J., Zhang, X., Zinkevich, M. (2017). Tfx: A tensorflow-based production-scale machine learning platform. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD \u201917, pp. 1387\u20131395. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3097983.3098021","DOI":"10.1145\/3097983.3098021"},{"key":"9601_CR5","unstructured":"Debois, P. (2011). DevOps: A software revolution in the making. Cutter IT Journal 24(8)."},{"key":"9601_CR6","unstructured":"Deloitte. (2017). Managing algorithmic risks \u2013 Safeguarding the use of complex algorithms and machine learning. https:\/\/www2.deloitte.com\/us\/en\/pages\/risk\/articles\/algorithmic-machine-learning-risk-management.html"},{"key":"9601_CR7","unstructured":"der Benannten Stellen f\u00fcr Medizinprodukte\u00a0in Deutschland\u00a0(IG-NB), I. (2021). Fragenkatalog K\u00fcnstliche Intelligenz bei Medizinprodukten. https:\/\/www.ig-nb.de\/dok_view?oid=861877. Retrieved 29 December\u00a02021."},{"key":"9601_CR8","unstructured":"Drvar, M., Turner, J., Piechocki, M., Stiegeler, E., M\u00fcnch, D. (2020). The future of data collection and data management: Agile RegOps for digitalizing the regulatory value chain. BearingPoint Software Solutions GmbH, Frankfurt."},{"issue":"3","key":"9601_CR9","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1109\/MS.2016.68","volume":"33","author":"C Ebert","year":"2016","unstructured":"Ebert, C., Gallardo, G., Hernantes, J., & Serrano, N. (2016). DevOps. IEEE Software, 33(3), 94\u2013100.","journal-title":"IEEE Software"},{"key":"#cr-split#-9601_CR10.1","unstructured":"European Parliament and the Council. (2017). Regulation"},{"key":"#cr-split#-9601_CR10.2","unstructured":"(EU) 2017\/745 on medical devices. https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/?uri=CELEX:02017R0745-20200424#tocId168. Retrieved 21 November\u00a02021."},{"key":"#cr-split#-9601_CR11.1","unstructured":"European Parliament and the Council. (2017). Regulation"},{"key":"#cr-split#-9601_CR11.2","unstructured":"(EU) 2017\/746 on in vitro diagnostic medical devices. https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/?uri=CELEX:02017R0746-20170505#tocId157. Retrieved 21 November\u00a02021."},{"key":"9601_CR12","unstructured":"FDA - Center for Devices and Radiological Health. (1997). Design Control Guidance for Medical Device Manufacturers."},{"key":"9601_CR13","doi-asserted-by":"publisher","first-page":"176","DOI":"10.1016\/j.jss.2015.06.063","volume":"123","author":"B Fitzgerald","year":"2017","unstructured":"Fitzgerald, B., & Stol, K.-J. (2017). Continuous software engineering: A roadmap and agenda. Journal of Systems and Software, 123, 176\u2013189.","journal-title":"Journal of Systems and Software"},{"key":"9601_CR14","unstructured":"Food and Drug Administration. (2021). Good Machine Learning Practice for Medical Device Development Guiding Principles. https:\/\/www.regulations.gov\/document\/FDA-2019-N-1185-0156"},{"key":"9601_CR15","unstructured":"Google Cloud Solutions. (2021). MLOps: Continuous delivery and automation pipelines in machine learning. Google Cloud. https:\/\/cloud.google.com\/solutions\/machine-learning\/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning. Accessed 14 March\u00a02021."},{"issue":"1","key":"9601_CR16","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1109\/MS.2021.3117292","volume":"39","author":"T Granlund","year":"2022","unstructured":"Granlund, T., Stirbu, V., & Mikkonen, T. (2022). Medical software needs calm compliance. IEEE Software, 39(1), 19\u201328. https:\/\/doi.org\/10.1109\/MS.2021.3117292","journal-title":"IEEE Software"},{"issue":"5","key":"9601_CR17","doi-asserted-by":"publisher","first-page":"342","DOI":"10.1007\/s42979-021-00726-1","volume":"2","author":"T Granlund","year":"2021","unstructured":"Granlund, T., Stirbu, V., & Mikkonen, T. (2021). Towards regulatory-compliant MLOps: Oravizio\u2019s journey from a machine learning experiment to a deployed certified medical product. SN Computer Science, 2(5), 342. https:\/\/doi.org\/10.1007\/s42979-021-00726-1","journal-title":"SN Computer Science"},{"key":"9601_CR18","doi-asserted-by":"publisher","unstructured":"Granlund, T., Vedenp\u00e4\u00e4, J., Stirbu, V., Mikkonen, T. (2021). On medical device cybersecurity compliance in eu. In: 2021 IEEE\/ACM 3rd International Workshop on Software Engineering for Healthcare (SEH), pp. 20\u201323. https:\/\/doi.org\/10.1109\/SEH52539.2021.00011","DOI":"10.1109\/SEH52539.2021.00011"},{"key":"9601_CR19","unstructured":"Humble, J., Farley, D. (2010). Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation. Pearson Education."},{"key":"9601_CR20","unstructured":"International Electrotechnical Commission. (2015). IEC 62304:2006\/A1:2015. Medical device software - Software life-cycle processes."},{"key":"9601_CR21","unstructured":"International Electrotechnical Commission. (2016). IEC 82304-1:2016. Health software - Part 1: General requirements for product safety."},{"key":"9601_CR22","unstructured":"International Medical Device Regulators Forum. (2022). Machine Learning-enabled Medical Devices: Key Terms and Definitions."},{"key":"9601_CR23","unstructured":"International Organization for Standardization. (2016). ISO 13485:2016. Medical devices - Quality management systems - Requirements for regulatory purposes."},{"key":"9601_CR24","unstructured":"International Organization for Standardization. (2019). ISO 14971:2019. Medical devices - Application of risk management to medical devices."},{"key":"9601_CR25","doi-asserted-by":"crossref","unstructured":"John, M. M., Olsson, H. H., Bosch, J. (2021). Towards MLOps: A framework and maturity model. In: 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA\u201921), pp. 1\u20138. IEEE.","DOI":"10.1109\/SEAA53835.2021.00050"},{"issue":"4","key":"9601_CR26","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1046\/j.1365-2575.2001.00113.x","volume":"11","author":"N J\u00f8rgensen","year":"2001","unstructured":"J\u00f8rgensen, N. (2001). Putting it all in the trunk: Incremental software development in the FreeBSD open source project. Information Systems Journal, 11(4), 321\u2013336.","journal-title":"Information Systems Journal"},{"issue":"3","key":"9601_CR27","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1145\/3236386.3241340","volume":"16","author":"ZC Lipton","year":"2018","unstructured":"Lipton, Z. C. (2018). The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3), 31\u201357.","journal-title":"Queue"},{"key":"9601_CR28","unstructured":"Medical Device Coordination Group. (2021). Ongoing guidance development and deliverables of MDCG Subgroups - October 2021. https:\/\/ec.europa.eu\/health\/sites\/default\/files\/md_sector\/docs\/ mdcg_ongoing_guidancedocs_en.pdf. Retrieved 29 December\u00a02021."},{"key":"9601_CR29","doi-asserted-by":"crossref","unstructured":"Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., Gebru, T. (2019). Model cards for model reporting. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 220\u2013229.","DOI":"10.1145\/3287560.3287596"},{"key":"9601_CR30","doi-asserted-by":"publisher","unstructured":"Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., Gebru, T. (2019). Model cards for model reporting. In: Proceedings of the Conference on Fairness, Accountability, and Transparency. FAT* \u201919, pp. 220\u2013229. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3287560.3287596","DOI":"10.1145\/3287560.3287596"},{"key":"9601_CR31","doi-asserted-by":"crossref","unstructured":"Myrbakken, H., Colomo-Palacios, R. (2017). DevSecOps: A multivocal literature review. In: International Conference on Software Process Improvement and Capability Determination, pp. 17\u201329. Springer.","DOI":"10.1007\/978-3-319-67383-7_2"},{"key":"9601_CR32","doi-asserted-by":"crossref","unstructured":"Rajkumar, M., Pole, A. K., Adige, V. S., Mahanta, P. (2016). Devops culture and its impact on cloud delivery and software development. In: 2016 International Conference on Advances in Computing, Communication, & Automation (ICACCA)(Spring), pp. 1\u20136. IEEE.","DOI":"10.1109\/ICACCA.2016.7578902"},{"key":"9601_CR33","unstructured":"Sato, D., Wilder, A., Windheuser, C. (2019). Continuous Delivery for Machine Learning. https:\/\/martinfowler.com\/articles\/cd4ml.html Retrieved 21 December\u00a02020."},{"key":"9601_CR34","unstructured":"Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.-F., Dennison, D. (2015). Hidden technical debt in machine learning systems. In: Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2. NIPS\u201915, pp. 2503\u20132511. MIT Press, Cambridge, MA, USA."},{"key":"9601_CR35","doi-asserted-by":"crossref","unstructured":"Stirbu, V., Mikkonen, T. (2021). Introducing traceability in github for medical software development. In: Product-Focused Software Process Improvement (PROFES\u201921). Springer","DOI":"10.1007\/978-3-030-91452-3_10"},{"key":"9601_CR36","doi-asserted-by":"crossref","unstructured":"Stirbu, V., Mikkonen, T. (2018). Towards agile yet regulatory-compliant development of medical software. In: 2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), pp. 337\u2013340. IEEE.","DOI":"10.1109\/ISSREW.2018.00027"},{"key":"9601_CR37","doi-asserted-by":"crossref","unstructured":"Taivalsaari, A., Mikkonen, T., Ingalls, D., Palacz, K. (2008). Web browser as an application platform. In: 2008 34th Euromicro Conference Software Engineering and Advanced Applications, pp. 293\u2013302. IEEE.","DOI":"10.1109\/SEAA.2008.17"},{"key":"9601_CR38","doi-asserted-by":"crossref","unstructured":"Toivakka, H., Granlund, T., Poranen, T., Zhang, Z. (2021). Towards RegOps: A DevOps Pipeline for Medical Device Software. In: Product Focused Software Improvement (PROFES\u201921). Springer.","DOI":"10.1007\/978-3-030-91452-3_20"},{"key":"9601_CR39","unstructured":"Treveil, M., Omont, N., Stenac, C., Lefevre, K., Phan, D., Zentici, J., Lavoillotte, A., Miyazaki, M., Heidmann, L. (2020). Introducing MLOps. O\u2019Reilly Media, Inc."},{"key":"9601_CR40","unstructured":"U.S. Department of Health and Human Services. (2021). Federal Food, Drug, and Cosmetic Act. https:\/\/www.fda.gov\/regulatory-information\/laws-enforced-fda\/federal-food-drug-and-cosmetic-act-fdc-act. Retrieved 21 November\u00a02021."},{"key":"9601_CR41","unstructured":"U.S. Food and Drug Administration (FDA). (2021). Artificial Intelligence\/Machine Learning (AI\/ML)-Based Software as a Medical Device (SaMD) Action Plan. https:\/\/www.fda.gov\/media\/145022\/download"},{"key":"9601_CR42","unstructured":"Vogel, D. A. (2011). Medical Device Software Verification, Validation and Compliance. Artech House, Boston\/London."}],"container-title":["Software Quality Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11219-022-09601-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11219-022-09601-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11219-022-09601-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,30]],"date-time":"2023-06-30T10:22:32Z","timestamp":1688120552000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11219-022-09601-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,17]]},"references-count":44,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,6]]}},"alternative-id":["9601"],"URL":"https:\/\/doi.org\/10.1007\/s11219-022-09601-5","relation":{},"ISSN":["0963-9314","1573-1367"],"issn-type":[{"value":"0963-9314","type":"print"},{"value":"1573-1367","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,17]]},"assertion":[{"value":"1 September 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 October 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 January 2023","order":3,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Update","order":4,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The original version of this article has been updated to correct the funding note.","order":5,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}