{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,20]],"date-time":"2025-12-20T22:11:08Z","timestamp":1766268668458,"version":"3.37.3"},"reference-count":69,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2022,3,25]],"date-time":"2022-03-25T00:00:00Z","timestamp":1648166400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,25]],"date-time":"2022-03-25T00:00:00Z","timestamp":1648166400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"BMBF","award":["01IS18091B"],"award-info":[{"award-number":["01IS18091B"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Innovations Syst Softw Eng"],"published-print":{"date-parts":[[2024,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Research on novel tools for model-based development differs from a mere engineering task by not only developing a new tool, but by providing some form of evidence that it is effective. This is typically achieved by experimental evaluations. Following principles of good scientific practice, both the tool and the models used in the experiments should be made available along with a paper, aiming at the replicability of experimental results. We investigate to which degree recent research reporting on novel methods, techniques, or algorithms supporting model-based development with MATLAB\/Simulink meets the requirements for replicability of experimental results. Our results from studying 65 research papers obtained through a systematic literature search are rather unsatisfactory. In a nutshell, we found that only 31% of the tools and 22% of the models used as experimental subjects are accessible. Given that both artifacts are needed for a replication study, only 9% of the tool evaluations presented in the examined papers can be classified to be replicable in principle. We found none of the experimental results presented in these papers to be fully replicable, and 6% partially replicable. Given that tools are still being listed among the major obstacles of a more widespread adoption of model-based principles in practice, we see this as an alarming signal. While we are convinced that this situation can only be improved as a community effort, this paper is meant to serve as starting point for discussion, based on the lessons learnt from our study.<\/jats:p>","DOI":"10.1007\/s11334-022-00442-w","type":"journal-article","created":{"date-parts":[[2022,5,6]],"date-time":"2022-05-06T21:12:35Z","timestamp":1651871555000},"page":"209-224","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Replicability of experimental tool evaluations in model-based software and systems engineering with MATLAB\/Simulink"],"prefix":"10.1007","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9881-9748","authenticated-orcid":false,"given":"Alexander","family":"Boll","sequence":"first","affiliation":[]},{"given":"Nicole","family":"Vieregg","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2582-5557","authenticated-orcid":false,"given":"Timo","family":"Kehrer","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,3,25]]},"reference":[{"issue":"1","key":"442_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/978-3-031-02549-5","volume":"3","author":"M Brambilla","year":"2017","unstructured":"Brambilla M, Cabot J, Wimmer M (2017) Model-driven software engineering in practice. Synth Lect Softw Eng 3(1):1\u2013207","journal-title":"Synth Lect Softw Eng"},{"key":"442_CR2","unstructured":"V\u00f6lter M, Stahl T, Bettin J, Haase A, Helsen S (2013) Model-driven software development: technology, engineering, management. John Wiley & Sons"},{"issue":"3","key":"442_CR3","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1109\/MS.2009.80","volume":"26","author":"P Liggesmeyer","year":"2009","unstructured":"Liggesmeyer P, Trapp M (2009) Trends in embedded software engineering. IEEE Softw 26(3):19\u201325","journal-title":"IEEE Softw"},{"key":"442_CR4","doi-asserted-by":"publisher","unstructured":"Robert France and Bernhard Rumpe, \u201cModel-driven Development of Complex Software: A Research Roadmap,\u201d Future of Software Engineering (FOSE \u201907), 2007, pp.\u00a037\u20134, https:\/\/doi.org\/10.1109\/FOSE.2007.14","DOI":"10.1109\/FOSE.2007.14"},{"issue":"1","key":"442_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10009-002-0083-4","volume":"4","author":"M Shaw","year":"2002","unstructured":"Shaw M (2002) What makes good research in software engineering? Int J Softw Tools Technol Transf 4(1):1\u20137","journal-title":"Int J Softw Tools Technol Transf"},{"issue":"5","key":"442_CR6","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1109\/2.675631","volume":"31","author":"WF Tichy","year":"1998","unstructured":"Tichy WF (1998) Should computer scientists experiment more? Computer 31(5):32\u201340","journal-title":"Computer"},{"key":"442_CR7","unstructured":"Meyer B (2010) Empirical research: questions from software engineering. In: 4th international symposium on empirical software engineering and measurement (ESEM 2010)"},{"key":"442_CR8","unstructured":"Barba LA (2018) Terminologies for reproducible research. arXiv preprint arXiv:1802.03311"},{"key":"442_CR9","doi-asserted-by":"publisher","unstructured":"Juristo N., G\u00f3mez O.S. (2012) Replication of Software Engineering Experiments. In: Meyer B., Nordio M. (eds) Empirical Software Engineering and Verification. LASER 2010, LASER 2009, LASER 2008. Lecture Notes in Computer Science, vol 7007. Springer, Berlin, Heidelberg. https:\/\/doi.org\/10.1007\/978-3-642-25231-0_2","DOI":"10.1007\/978-3-642-25231-0_2"},{"issue":"4","key":"442_CR10","doi-asserted-by":"publisher","first-page":"456","DOI":"10.1109\/32.799939","volume":"25","author":"VR Basili","year":"1999","unstructured":"Basili VR, Shull F, Lanubile F (1999) Building knowledge through families of experiments. IEEE Trans Softw Eng 25(4):456\u2013473","journal-title":"IEEE Trans Softw Eng"},{"key":"442_CR11","doi-asserted-by":"publisher","unstructured":"Mark D. Wilkinson., Michel Dumontier, IJsbrand J. Aalbersberg, et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 3, 160018 (2016). https:\/\/doi.org\/10.1038\/sdata.2016.18","DOI":"10.1038\/sdata.2016.18"},{"key":"442_CR12","doi-asserted-by":"crossref","unstructured":"Lamprecht AL, Garcia L, Kuzak M, Martinez C, Arcila R, Martin Del\u00a0Pico E, Dominguez Del\u00a0Angel V, van\u00a0de Sandt S, Ison J, Martinez PA, et\u00a0al. (2019) Towards fair principles for research software. Data Sci pp. 1\u201323","DOI":"10.3233\/DS-190026"},{"issue":"3","key":"442_CR13","doi-asserted-by":"publisher","first-page":"e308","DOI":"10.1371\/journal.pone.0000308","volume":"2","author":"HA Piwowar","year":"2007","unstructured":"Piwowar HA, Day RS, Fridsma DB (2007) Sharing detailed research data is associated with increased citation rate. PloS One 2(3):e308","journal-title":"PloS One"},{"key":"442_CR14","doi-asserted-by":"publisher","first-page":"e175","DOI":"10.7717\/peerj.175","volume":"1","author":"HA Piwowar","year":"2013","unstructured":"Piwowar HA, Vision TJ (2013) Data reuse and the open data citation advantage. PeerJ 1:e175","journal-title":"PeerJ"},{"key":"442_CR15","doi-asserted-by":"crossref","unstructured":"Whittle J, Hutchinson J, Rouncefield M, Burden H, Heldal R (2013) Industrial adoption of model-driven engineering: Are the tools really the problem? In: International Conference on Model Driven Engineering Languages and Systems. pp. 1\u201317. Springer","DOI":"10.1007\/978-3-642-41533-3_1"},{"key":"442_CR16","doi-asserted-by":"crossref","unstructured":"Boll A, Kehrer T (2020) On the replicability of experimental tool evaluations in model-based development. In: International conference on systems modelling and management. Springer, pp. 111\u2013130","DOI":"10.1007\/978-3-030-58167-1_9"},{"key":"442_CR17","unstructured":"Barbara Kitchenham, Stuart Charters, Guidelines for performing Systematic Literature Reviews in Software Engineering, Technical Report, Version 2.3, Keele University and University of Durham, 9 July 2007"},{"key":"442_CR18","doi-asserted-by":"crossref","unstructured":"Rebaya A, Gasmi K, Hasnaoui S (2018) A Simulink-based rapid prototyping workflow for optimizing software\/hardware programming. In: 2018 26th International Conference on Software, Telecommunications and Computer Networks (SoftCOM). pp.\u00a01\u20136. IEEE","DOI":"10.23919\/SOFTCOM.2018.8555777"},{"key":"442_CR19","doi-asserted-by":"publisher","unstructured":"Kuroki Y, Yoo M, Yokoyama T (2016) A Simulink to UML model transformation tool for embedded control software development. In: IEEE international conference on industrial technology, ICIT 2016, Taipei, Taiwan. IEEE, pp. 700\u2013706. https:\/\/doi.org\/10.1109\/ICIT.2016.7474835","DOI":"10.1109\/ICIT.2016.7474835"},{"key":"442_CR20","doi-asserted-by":"crossref","unstructured":"Stephan M, Cordy JR (2015) Identifying instances of model design patterns and antipatterns using model clone detection. In: Proceedings of the seventh international workshop on modeling in software engineering. MiSE \u201915, IEEE Press, pp. 48\u201353","DOI":"10.1109\/MiSE.2015.16"},{"key":"442_CR21","doi-asserted-by":"publisher","unstructured":"Matinnejad R, Nejati S, Briand LC, Bruckmann T (May 2016) Automated test suite generation for time-continuous Simulink models. In: 2016 IEEE\/ACM 38th international conference on software engineering (ICSE), pp. 595\u2013606. https:\/\/doi.org\/10.1145\/2884781.2884797","DOI":"10.1145\/2884781.2884797"},{"issue":"9","key":"442_CR22","doi-asserted-by":"publisher","first-page":"919","DOI":"10.1109\/TSE.2018.2811489","volume":"45","author":"R Matinnejad","year":"2019","unstructured":"Matinnejad R, Nejati S, Briand LC, Bruckmann T (2019) Test generation and test prioritization for Simulink models with dynamic behavior. IEEE Trans Softw Eng 45(9):919\u2013944. https:\/\/doi.org\/10.1109\/TSE.2018.2811489","journal-title":"IEEE Trans Softw Eng"},{"key":"442_CR23","doi-asserted-by":"publisher","unstructured":"Nejati S, Gaaloul K, Menghi C, Briand LC, Foster S, Wolfe D (2019) Evaluating model testing and model checking for finding requirements violations in Simulink models. In: Proceedings of the 2019 27th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering. ESEC\/FSE 2019, Association for computing machinery, New York, NY, USA, pp. 1015\u20131025. https:\/\/doi.org\/10.1145\/3338906.3340444","DOI":"10.1145\/3338906.3340444"},{"key":"442_CR24","doi-asserted-by":"publisher","unstructured":"Rao AC, Raouf A, Dhadyalla G, Pasupuleti V (2017) Mutation testing based evaluation of formal verification tools. In: 2017 international conference on dependable systems and their applications (DSA), pp.\u00a01\u20137. https:\/\/doi.org\/10.1109\/DSA.2017.10","DOI":"10.1109\/DSA.2017.10"},{"key":"442_CR25","doi-asserted-by":"publisher","unstructured":"Gerlitz T, Kowalewski S (2016) Flow sensitive slicing for MATLAB\/Simulink models. In: 2016 13th working IEEE\/IFIP conference on software architecture (WICSA), pp. 81\u201390. https:\/\/doi.org\/10.1109\/WICSA.2016.23","DOI":"10.1109\/WICSA.2016.23"},{"key":"442_CR26","doi-asserted-by":"publisher","unstructured":"Khelifi A, Ben Lakhal NM, Gharsallaoui H, Nasri O (2018) Artificial neural network-based fault detection. In: 2018 5th international conference on control, decision and information technologies (CoDIT), pp. 1017\u20131022. https:\/\/doi.org\/10.1109\/CoDIT.2018.8394963","DOI":"10.1109\/CoDIT.2018.8394963"},{"key":"442_CR27","doi-asserted-by":"publisher","DOI":"10.1016\/j.matpr.2019.07.689","author":"O Oussalem","year":"2019","unstructured":"Oussalem O, Kourchi M, Rachdy A, Ajaamoum M, Idadoub H, Jenkal S (2019) A low cost controller of PV system based on Arduino board and INC algorithm. Mater Today Proc. https:\/\/doi.org\/10.1016\/j.matpr.2019.07.689","journal-title":"Mater Today Proc"},{"key":"442_CR28","unstructured":"Norouzi P, K\u0131van\u00e7 \u00d6C, \u00dcst\u00fcn \u00d6 (2017) High performance position control of double sided air core linear brushless DC motor. In: 2017 10th international conference on electrical and electronics engineering (ELECO), pp. 233\u2013238"},{"key":"442_CR29","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1016\/j.engstruct.2015.09.008","volume":"103","author":"J Gallego-Calderon","year":"2015","unstructured":"Gallego-Calderon J, Natarajan A (2015) Assessment of wind turbine drive-train fatigue loads under torsional excitation. Eng Struct 103:189\u2013202. https:\/\/doi.org\/10.1016\/j.engstruct.2015.09.008","journal-title":"Eng Struct"},{"key":"442_CR30","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1016\/j.jss.2015.04.089","volume":"106","author":"M Rashid","year":"2015","unstructured":"Rashid M, Anwar MW, Khan AM (2015) Toward the tools selection in model based system engineering for embedded systems\u2014a systematic literature review. J Syst Softw 106:150\u2013163","journal-title":"J Syst Softw"},{"key":"442_CR31","doi-asserted-by":"publisher","unstructured":"Elberzhager F, Rosbach A, Bauer T (2013) Analysis and testing of Matlab simulink models: a systematic mapping study. In: Proceedings of the 2013 international workshop on joining AcadeMiA and industry contributions to testing automation. JAMAICA 2013, association for computing machinery, New York, NY, USA, pp. 29\u201334. https:\/\/doi.org\/10.1145\/2489280.2489285","DOI":"10.1145\/2489280.2489285"},{"key":"442_CR32","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1002\/jrsm.1378","volume":"11","author":"M Gusenbauer","year":"2019","unstructured":"Gusenbauer M, Haddaway NR (2019) Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed and 26 other resources. Res Synth Methods 11:181\u2013217","journal-title":"Res Synth Methods"},{"issue":"1","key":"442_CR33","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1177\/001316446002000104","volume":"20","author":"J Cohen","year":"1960","unstructured":"Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37\u201346","journal-title":"Educ Psychol Meas"},{"key":"442_CR34","doi-asserted-by":"publisher","unstructured":"Yakimenko OA (2019) Engineering computations and modeling in MATLAB\u00ae\/Simulink\u00ae. American Institute of Aeronautics and Astronautics, Inc 12700 Sunrise Valley Drive, Suite 200 Reston, VA 20191-5807 ISBN (print): 978-1-62410-515-9 https:\/\/doi.org\/10.2514\/4.105159","DOI":"10.2514\/4.105159"},{"issue":"4","key":"442_CR35","doi-asserted-by":"publisher","first-page":"833","DOI":"10.1016\/j.joi.2013.08.002","volume":"7","author":"L Waltman","year":"2013","unstructured":"Waltman L, van Eck NJ (2013) A systematic empirical comparison of different approaches for normalizing citation impact indicators. J Informetr 7(4):833\u2013849","journal-title":"J Informetr"},{"key":"442_CR36","doi-asserted-by":"publisher","unstructured":"Boll A, Vieregg N, Kehrer T, The download link of digital artifacts of this paper, for reuse and replication. https:\/\/doi.org\/10.6084\/m9.figshare.13633928","DOI":"10.6084\/m9.figshare.13633928"},{"key":"442_CR37","doi-asserted-by":"publisher","DOI":"10.1145\/3126516","author":"T Bourke","year":"2017","unstructured":"Bourke T, Carcenac F, Cola\u00e7o JL, Pagano B, Pasteur C, Pouzet M (2017) A synchronous look at the simulink standard library. ACM Trans Embed Comput Syst. https:\/\/doi.org\/10.1145\/3126516","journal-title":"ACM Trans Embed Comput Syst"},{"key":"442_CR38","doi-asserted-by":"publisher","unstructured":"Chowdhury SA, Mohian S, Mehra S, Gawsane S, Johnson TT, Csallner C (2018) Automatically finding bugs in a commercial cyber-physical system development tool chain with SLforge. In: 2018 IEEE\/ACM 40th international conference on software engineering (ICSE), pp. 981\u2013992. https:\/\/doi.org\/10.1145\/3180155.3180231","DOI":"10.1145\/3180155.3180231"},{"key":"442_CR39","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-015-0477-x","author":"P Bostr\u00f6m","year":"2015","unstructured":"Bostr\u00f6m P, Wiik J (2015) Contract-based verification of discrete-time multi-rate simulink models. Softw Syst Model. https:\/\/doi.org\/10.1007\/s10270-015-0477-x","journal-title":"Softw Syst Model"},{"key":"442_CR40","doi-asserted-by":"publisher","unstructured":"Arrieta A, Wang S, Arruabarrena A, Markiegi U, Sagardui G, Etxeberria L (2018) Multi-objective black-box test case selection for cost-effectively testing simulation models. In: Proceedings of the genetic and evolutionary computation conference. GECCO \u201918, association for computing machinery, New York, NY, USA, p. 1411 \u2013 1418. https:\/\/doi.org\/10.1145\/3205455.3205490","DOI":"10.1145\/3205455.3205490"},{"key":"442_CR41","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1016\/j.infsof.2019.06.009","volume":"114","author":"A Arrieta","year":"2019","unstructured":"Arrieta A, Wang S, Markiegi U, Arruabarrena A, Etxeberria L, Sagardui G (2019) Pareto efficient multi-objective black-box test case selection for simulation-based testing. Inf Softw Technol 114:137\u2013154. https:\/\/doi.org\/10.1016\/j.infsof.2019.06.009","journal-title":"Inf Softw Technol"},{"key":"442_CR42","doi-asserted-by":"publisher","unstructured":"Mancini T, Mari F, Massini A, Melatti I, Tronci E (2015) Sylvaas: system level formal verification as a service. In: 2015 23rd euromicro international conference on parallel, distributed, and network-based processing, pp. 476\u2013483. https:\/\/doi.org\/10.1109\/PDP.2015.119","DOI":"10.1109\/PDP.2015.119"},{"key":"442_CR43","doi-asserted-by":"publisher","unstructured":"Morozov A, Ding K, Chen T, Janschek K (2017) Test suite prioritization for efficient regression testing of model-based automotive software. In: 2017 international conference on software analysis, testing and evolution (SATE), pp. 20\u201329. https:\/\/doi.org\/10.1109\/SATE.2017.11","DOI":"10.1109\/SATE.2017.11"},{"key":"442_CR44","doi-asserted-by":"publisher","unstructured":"Holling D, Hofbauer A, Pretschner A, Gemmar M (2016) Profiting from unit tests for integration testing. In: 2016 IEEE international conference on software testing, verification and validation (ICST), pp. 353\u2013363. https:\/\/doi.org\/10.1109\/ICST.2016.28","DOI":"10.1109\/ICST.2016.28"},{"key":"442_CR45","unstructured":"Strathmann T, Oehlerking J (2015) Verifying properties of an electro-mechanical braking system. In: 2nd workshop on applied verification of continuous and hybrid systems (ARCH 2015)"},{"key":"442_CR46","doi-asserted-by":"publisher","unstructured":"Bertram V, Maoz S, Ringert JO, Rumpe B, von Wenckstern M (2017) Component and connector views in practice: An experience report. In: Proceedings of the ACM\/IEEE 20th International Conference on Model Driven Engineering Languages and Systems. p. 167\u2013177. MODELS \u201917, IEEE Press. https:\/\/doi.org\/10.1109\/MODELS.2017.29","DOI":"10.1109\/MODELS.2017.29"},{"key":"442_CR47","doi-asserted-by":"publisher","unstructured":"Kusmenko E, Shumeiko I, Rumpe B, von Wenckstern M (2018) Fast simulation preorder algorithm. In: Proceedings of the 6th international conference on model-driven engineering and software development. MODELSWARD 2018, SCITEPRESS - Science and Technology Publications, Lda, Setubal, PRT. https:\/\/doi.org\/10.5220\/0006722102560267","DOI":"10.5220\/0006722102560267"},{"key":"442_CR48","doi-asserted-by":"crossref","unstructured":"Bertram V, Maoz S, Ringert JO, Rumpe B, von Wenckstern M (2017) Component and connector views in practice: an experience report. In: 2017 ACM\/IEEE 20th International conference on model driven engineering languages and systems (MODELS), IEEE, pp. 167\u2013177","DOI":"10.1109\/MODELS.2017.29"},{"issue":"8","key":"442_CR49","doi-asserted-by":"publisher","first-page":"792","DOI":"10.1016\/j.infsof.2010.03.006","volume":"52","author":"B Kitchenham","year":"2010","unstructured":"Kitchenham B, Pretorius R, Budgen D, Brereton OP, Turner M, Niazi M, Linkman S (2010) Systematic literature reviews in software engineering-a tertiary study. Inf Softw Technol 52(8):792\u2013805","journal-title":"Inf Softw Technol"},{"key":"442_CR50","doi-asserted-by":"publisher","unstructured":"Fr\u00e2ncila Weidt and Rodrigo Silva, Systematic literature review in computer science-a practical guide, Technical Report, Federal University of Juiz de Fora, November 2016, https:\/\/doi.org\/10.13140\/RG.2.2.35453.87524","DOI":"10.13140\/RG.2.2.35453.87524"},{"key":"442_CR51","unstructured":"Stapi\u0107 Z, L\u00f3pez EG, Cabot AG, de\u00a0Marcos\u00a0Ortega L, Strahonja V (2012) Performing systematic literature review in software engineering. In: CECIIS 2012-23rd international conference"},{"key":"442_CR52","doi-asserted-by":"crossref","unstructured":"Masuzzo P, Martens L (2017) Do you speak open science? resources and tips to learn the language. Tech. rep, PeerJ Preprints","DOI":"10.7287\/peerj.preprints.2689v1"},{"key":"442_CR53","doi-asserted-by":"publisher","unstructured":"Tomita T, Ishii D, Murakami T, Takeuchi S, Aoki T (2019) A scalable Monte-Carlo test-case generation tool for large and complex simulink models. In: 2019 IEEE\/ACM 11th International Workshop on Modelling in Software Engineering (MiSE). pp. 39\u201346. https:\/\/doi.org\/10.1109\/MiSE.2019.00014","DOI":"10.1109\/MiSE.2019.00014"},{"key":"442_CR54","doi-asserted-by":"publisher","first-page":"149422","DOI":"10.1109\/ACCESS.2019.2946747","volume":"7","author":"A Hussain","year":"2019","unstructured":"Hussain A, Sher HA, Murtaza AF, Al-Haddad K (2019) Improved restricted control set model predictive control (iRCS-MPC) based maximum power point tracking of photovoltaic module. IEEE Access 7:149422\u2013149432. https:\/\/doi.org\/10.1109\/ACCESS.2019.2946747","journal-title":"IEEE Access"},{"key":"442_CR55","doi-asserted-by":"publisher","unstructured":"Jiang Z, Wu X, Dong Z, Mu M Optimal test case generation for Simulink models using slicing. In: 2017 IEEE international conference on software quality, reliability and security companion (QRS-C), pp. 363\u2013369. https:\/\/doi.org\/10.1109\/QRS-C.2017.67","DOI":"10.1109\/QRS-C.2017.67"},{"key":"442_CR56","doi-asserted-by":"crossref","unstructured":"Chowdhury SA (2018) Understanding and improving cyber-physical system models and development tools. In: 2018 IEEE\/ACM 40th international conference on software engineering: companion (ICSE-Companion), pp. 452\u2013453","DOI":"10.1145\/3183440.3183455"},{"key":"442_CR57","doi-asserted-by":"crossref","unstructured":"Kehrer T, Kelter U, Pietsch P, Schmidt M (2012) Adaptability of model comparison tools. In: Proceedings of the 27th IEEE\/ACM international conference on automated software engineering. IEEE, pp. 306\u2013309","DOI":"10.1145\/2351676.2351731"},{"key":"442_CR58","doi-asserted-by":"crossref","unstructured":"Kehrer T, Kelter U, Ohrndorf M, Sollbach T (2012) Understanding model evolution through semantically lifting model differences with SiLift. In: 28th IEEE international conference on software maintenance (ICSM). IEEE, pp. 638\u2013641","DOI":"10.1109\/ICSM.2012.6405342"},{"key":"442_CR59","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1016\/j.scico.2018.04.002","volume":"163","author":"D Wille","year":"2018","unstructured":"Wille D, Babur \u00d6, Cleophas L, Seidl C, van den Brand M, Schaefer I (2018) Improving custom-tailored variability mining using outlier and cluster detection. Sci Comput Program 163:62\u201384. https:\/\/doi.org\/10.1016\/j.scico.2018.04.002","journal-title":"Sci Comput Program"},{"key":"442_CR60","doi-asserted-by":"crossref","unstructured":"Chowdhury SA, Varghese LS, Mohian S, Johnson TT, Csallner C (2018) A curated corpus of Simulink models for model-based empirical studies. In: 2018 IEEE\/ACM 4th international workshop on software engineering for smart cyber-physical systems (SEsCPS). IEEE, pp. 45\u201348","DOI":"10.1145\/3196478.3196484"},{"key":"442_CR61","doi-asserted-by":"crossref","unstructured":"Chowdhury SA, Shrestha SL, Johnson TT, Csallner C (2020) SLEMI: Equivalence modulo input (EMI) based mutation of CPS models for finding compiler bugs in Simulink. In: Proceeding 42nd ACM\/IEEE international conference on software engineering (ICSE), ACM. To appear","DOI":"10.1145\/3377811.3380381"},{"key":"442_CR62","doi-asserted-by":"crossref","unstructured":"Sanchez B, Zolotas A, Rodriguez HH, Kolovos D, Paige R (2019) On-the-fly translation and execution of OCL-like queries on simulink models. In: 2019 ACM\/IEEE 22nd international conference on model driven engineering languages and systems (MODELS). IEEE, pp. 205\u2013215","DOI":"10.1109\/MODELS.2019.000-1"},{"key":"442_CR63","unstructured":"Bourbouh H, Garoche PL, Garion C, Gurfinkel A, Kahsai T, Thirioux X (2017) Automated analysis of stateflow models. In: 21st International conference on logic for programming, artificial intelligence and reasoning (LPAR 2017), pp. 144\u2013161"},{"key":"442_CR64","unstructured":"Ernst G, Arcaini P, Donze A, Fainekos G, Mathesen L, Pedrielli G, Yaghoubi S, Yamagata Y, Zhang Z (2019) Arch-comp 2019 category report: falsification. In: ARCH@ CPSIoTWeek, pp. 129\u2013140"},{"key":"442_CR65","doi-asserted-by":"crossref","unstructured":"Dajsuren Y, van\u00a0den Brand MG, Serebrenik A, Roubtsov S (2013) Simulink models are also software: modularity assessment. In: 9th international ACM sigsoft conference on quality of software architectures (QoSA), pp. 99\u2013106","DOI":"10.1145\/2465478.2465482"},{"key":"442_CR66","unstructured":"Dajsuren Y (2015) On the design of an architecture framework and quality evaluation for automotive software systems. Ph.D. thesis, Department of mathematics and computer science, Technische Universiteit Eindhoven"},{"key":"442_CR67","doi-asserted-by":"crossref","unstructured":"Hebig R, Quang TH, Chaudron MR, Robles G, Fernandez MA (2016) The quest for open source projects that use UML: mining GitHub. In: Proceedings of the ACM\/IEEE 19th international conference on model driven engineering languages and systems, pp. 173\u2013183","DOI":"10.1145\/2976767.2976778"},{"key":"442_CR68","unstructured":"Philip Langer, Tanja Mayerhofer, Manuel Wimmer, Gerti Kappel, On the Usage of UML: Initial Results of Analyzing Open UML Models, Modellierung 2014, Editors: Hans-Georg Fill, Dimitris Karagiannis, Ulrich Reimer, Gesellschaft f\u00fcr Informatik e.V., Bonn, 2014 ISBN 978-388579-619-0 http:\/\/dl.gi.de\/handle\/20.500.12116\/20950"},{"key":"442_CR69","doi-asserted-by":"crossref","unstructured":"Boll A, Brokhausen F, Amorim T, Kehrer T, Vogelsang A, Characteristics, potentials, and limitations of open-source simulink projects for empirical research. Softw Syst Model pp. 1\u201320 (2021)","DOI":"10.1007\/s10270-021-00883-0"}],"container-title":["Innovations in Systems and Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11334-022-00442-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11334-022-00442-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11334-022-00442-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,30]],"date-time":"2024-07-30T10:08:41Z","timestamp":1722334121000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11334-022-00442-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,25]]},"references-count":69,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,9]]}},"alternative-id":["442"],"URL":"https:\/\/doi.org\/10.1007\/s11334-022-00442-w","relation":{},"ISSN":["1614-5046","1614-5054"],"issn-type":[{"type":"print","value":"1614-5046"},{"type":"electronic","value":"1614-5054"}],"subject":[],"published":{"date-parts":[[2022,3,25]]},"assertion":[{"value":"25 January 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 February 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 March 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}