{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T21:31:24Z","timestamp":1767994284294,"version":"3.49.0"},"reference-count":57,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2023,9,1]],"date-time":"2023-09-01T00:00:00Z","timestamp":1693526400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,9,18]],"date-time":"2023-09-18T00:00:00Z","timestamp":1694995200000},"content-version":"vor","delay-in-days":17,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100006211","name":"Humboldt-Universit\u00e4t zu Berlin","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100006211","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2023,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Context<\/jats:title>\n                <jats:p>Ensuring safety for any sophisticated system is getting more complex due to the rising number of features and functionalities. This calls for formal methods to entrust confidence in such systems. Nevertheless, using formal methods in industry is demanding because of their lack of usability and the difficulty of understanding verification results.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Objective<\/jats:title>\n                <jats:p>We evaluate the acceptance of formal methods by Bosch automotive engineers, particularly whether the difficulty of understanding verification results can be reduced.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Method<\/jats:title>\n                <jats:p>We perform two different exploratory studies. First, we conduct a user survey to explore challenges in identifying inconsistent specifications and using formal methods by Bosch automotive engineers. Second, we perform a one-group pretest-posttest experiment to collect impressions from Bosch engineers familiar with formal methods to evaluate whether understanding verification results is simplified by our counterexample explanation approach.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The results from the user survey indicate that identifying refinement inconsistencies, understanding formal notations, and interpreting verification results are challenging. Nevertheless, engineers are still interested in using formal methods in real-world development processes because it could reduce the manual effort for verification. Additionally, they also believe formal methods could make the system safer. Furthermore, the one-group pretest-posttest experiment results indicate that engineers are more comfortable understanding the counterexample explanation than the raw model checker output.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Limitations<\/jats:title>\n                <jats:p>The main limitation of this study is the generalizability beyond the target group of Bosch automotive engineers.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s10664-023-10353-4","type":"journal-article","created":{"date-parts":[[2023,9,18]],"date-time":"2023-09-18T10:01:57Z","timestamp":1695031317000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["A user study for evaluation of formal verification results and their explanation at Bosch"],"prefix":"10.1007","volume":"28","author":[{"given":"Arut Prakash","family":"Kaleeswaran","sequence":"first","affiliation":[]},{"given":"Arne","family":"Nordmann","sequence":"additional","affiliation":[]},{"given":"Thomas","family":"Vogel","sequence":"additional","affiliation":[]},{"given":"Lars","family":"Grunske","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,9,18]]},"reference":[{"key":"10353_CR1","doi-asserted-by":"publisher","unstructured":"Abrial J (2006) Formal methods in industry: achievements, problems, future. In: Osterweil LJ, Rombach HD, Soffa ML (eds) 28th International Conference on Software Engineering (ICSE 2006)May 20-28, 2006. Shanghai, China, p 761\u2013768 ACM https:\/\/doi.org\/10.1145\/1134285.1134406","DOI":"10.1145\/1134285.1134406"},{"issue":"7","key":"10353_CR2","doi-asserted-by":"publisher","first-page":"620","DOI":"10.1109\/TSE.2015.2398877","volume":"41","author":"M Autili","year":"2015","unstructured":"Autili M, Grunske L, Lumpe M, Pelliccione P, Tang A (2015) Aligning qualitative, real-time, and probabilistic property specification patterns using a structured english grammar. IEEE Trans. Software Eng. 41(7):620\u2013638. https:\/\/doi.org\/10.1109\/TSE.2015.2398877","journal-title":"IEEE Trans. Software Eng."},{"key":"10353_CR3","unstructured":"Babbie ER (2016) The basics of social research. Cengage learning"},{"key":"10353_CR4","unstructured":"Baier C, Katoen J (2008) Principles of model checking. MIT Press. ISBN 978-0-262-02649-9"},{"key":"10353_CR5","doi-asserted-by":"publisher","unstructured":"Barbon G, Leroy V, Sala\u00fcn G (2019) Debugging of behavioural models with CLEAR. In: Vojnar T, Zhang L (eds) Tools and Algorithms for the Construction and Analysis of Systems - 25th International Conference, TACAS 2019, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2019, Prague, Czech Republic, April 6-11, 2019, Proceedings, Part I vol 11427. Lecture Notes in Computer Science. Springer, pp 386\u2013392. https:\/\/doi.org\/10.1007\/978-3-030-17462-0_26","DOI":"10.1007\/978-3-030-17462-0_26"},{"key":"10353_CR6","doi-asserted-by":"publisher","unstructured":"Bicarregui J, Fitzgerald JS, Larsen PG, Woodcock JCP (2009) Industrial practice in formal methods: A review. In: Cavalcanti A, Dams D(eds) FM 2009: Formal Methods, Second World Congress, Eindhoven, The Netherlands, November 2-6, 2009. Proceedings, vol 5850 of Lecture Notes in Computer Science. Springer, pp 810\u2013813. https:\/\/doi.org\/10.1007\/978-3-642-05089-3_52","DOI":"10.1007\/978-3-642-05089-3_52"},{"key":"10353_CR7","doi-asserted-by":"publisher","unstructured":"Bowen JP, Breuer PT (2021) Formal methods communities of practice: A survey of personal experience. In: Cerone A, Autili M, Bucaioni A, Gomes C, Graziani P, Palmieri M, Temperini M, Venture G (eds) Software Engineering and Formal Methods. SEFM 2021 Collocated Workshops - CIFMA, CoSim-CPS, OpenCERT, ASYDE, Virtual Event, December 6-10, 2021, Revised Selected Papers vol 13230 of Lecture Notes in Computer Science. Springer, pp 287\u2013301. https:\/\/doi.org\/10.1007\/978-3-031-12429-7_21","DOI":"10.1007\/978-3-031-12429-7_21"},{"key":"10353_CR8","doi-asserted-by":"publisher","unstructured":"Bozzano M, Munk P, Schweizer M, Tonetta S, Voz\u00e1rov\u00e1 V (2020) Model-based safety analysis of mode transitions. In: Casimiro A, Ortmeier F, Bitsch F, Ferreira P (eds) Computer Safety, Reliability, and Security - 39th International Conference, SAFECOMP 2020, Lisbon, Portugal, September 16-18, 2020, Proceedings vol 12234 of Lecture Notes in Computer Science. Springer, pp 99\u2013114. https:\/\/doi.org\/10.1007\/978-3-030-54549-9_7","DOI":"10.1007\/978-3-030-54549-9_7"},{"key":"10353_CR9","unstructured":"Campbell DT, Stanley JC (1963) Experimental and quasi-experimental designs for research. Rand McNally Chicago"},{"key":"10353_CR10","doi-asserted-by":"crossref","unstructured":"Cimatti A, Tonetta S (2012) A property-based proof system for contract-based design. In: 38th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2012, Cesme, Izmir, Turkey, September 5-8, 2012. pp 21\u201328","DOI":"10.1109\/SEAA.2012.68"},{"issue":"4","key":"10353_CR11","doi-asserted-by":"publisher","first-page":"410","DOI":"10.1007\/s100090050046","volume":"2","author":"A Cimatti","year":"2000","unstructured":"Cimatti A, Clarke EM, Giunchiglia F, Roveri M (2000) NUSMV: A new symbolic model checker. Int J Softw Tools Technol Transf 2(4):410\u2013425. https:\/\/doi.org\/10.1007\/s100090050046","journal-title":"Int J Softw Tools Technol Transf"},{"issue":"4","key":"10353_CR12","doi-asserted-by":"publisher","first-page":"626","DOI":"10.1145\/242223.242257","volume":"28","author":"EM Clarke","year":"1996","unstructured":"Clarke EM, Wing JM (1996) Formal methods: State of the art and future directions. ACM Comput Surv 28(4):626\u2013643. https:\/\/doi.org\/10.1145\/242223.242257","journal-title":"ACM Comput Surv"},{"key":"10353_CR13","unstructured":"Clarke EM, Grumberg O, Kroening D, Peled DA, Veith H (2018a) Model checking, 2nd Edition. MIT Press. ISBN 978-0-262-03883-6"},{"key":"10353_CR14","doi-asserted-by":"publisher","unstructured":"Clarke EM, Henzinger TA, Veith H, Bloem R (eds) (2018b) Handbook of Model Checking. Springer. ISBN 978-3-319-10574-1.https:\/\/doi.org\/10.1007\/978-3-319-10575-8","DOI":"10.1007\/978-3-319-10575-8"},{"key":"10353_CR15","doi-asserted-by":"publisher","unstructured":"Davis JA, Clark MA, Cofer DD, Fifarek A, Hinchman J, Hoffman JA, Hulbert BW, Miller SP, Wagner LG (2013) Study on the barriers to the industrial adoption of formal methods. In: Pecheur C, Dierkes M (eds) Formal Methods for Industrial Critical Systems - 18th International Workshop, FMICS 2013, Madrid, Spain, September 23-24, 2013. Proceedings, vol 8187 of Lecture Notes in Computer Science. Springer, pp 63\u201377. https:\/\/doi.org\/10.1007\/978-3-642-41010-9_5","DOI":"10.1007\/978-3-642-41010-9_5"},{"key":"10353_CR16","doi-asserted-by":"publisher","unstructured":"de\u00a0Moura LM, Bj\u00f8rner N (2008) Z3: an efficient SMT solver. In: Ramakrishnan CR, Rehof J (eds) Tools and Algorithms for the Construction and Analysis of Systems, 14th International Conference, TACAS 2008, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2008, Budapest, Hungary, March 29-April 6, 2008. Proceedings, vol 4963 of Lecture Notes in Computer Science. Springer, pages 337\u2013340. https:\/\/doi.org\/10.1007\/978-3-540-78800-3_24","DOI":"10.1007\/978-3-540-78800-3_24"},{"key":"10353_CR17","doi-asserted-by":"publisher","unstructured":"Dwyer MB, Avrunin GS, Corbett JC (1999) Patterns in property specifications for finite-state verification. In: Boehm BW, Garlan D, Kramer J (eds) Proceedings of the 1999 International Conference on Software Engineering, ICSE\u2019 99, Los Angeles, CA, USA, May 16-22, 1999. ACM. pp 411\u2013420. https:\/\/doi.org\/10.1145\/302405.302672","DOI":"10.1145\/302405.302672"},{"key":"10353_CR18","doi-asserted-by":"publisher","unstructured":"Ferrari A, ter Beek MH (2023) Formal methods in railways: A systematic mapping study. ACM Comput Surv 55(4):69:1\u201369:37.https:\/\/doi.org\/10.1145\/3520480","DOI":"10.1145\/3520480"},{"key":"10353_CR19","doi-asserted-by":"publisher","unstructured":"Ferrari A, ter Beek MH, Mazzanti F, Basile D, Fantechi A, Gnesi S, Piattino A, Trentini D (2019) Survey on formal methods and tools in railways: The astrail approach. In: Dutilleul SC, Lecomte T, Romanovsky AB (eds0) Reliability, Safety, and Security of Railway Systems. Modelling, Analysis, Verification, and Certification - Third International Conference, RSSRail 2019, Lille, France, June 4-6, 2019, Proceedings, vol 11495 of Lecture Notes in Computer Science. Springer, pp 226\u2013241. https:\/\/doi.org\/10.1007\/978-3-030-18744-6_15","DOI":"10.1007\/978-3-030-18744-6_15"},{"key":"10353_CR20","doi-asserted-by":"crossref","unstructured":"Fink A (2003) The survey handbook. sage","DOI":"10.4135\/9781412986328"},{"key":"10353_CR21","volume-title":"A practical guide to SysML: the systems modeling language","author":"S Friedenthal","year":"2014","unstructured":"Friedenthal S, Moore A, Steiner R (2014) A practical guide to SysML: the systems modeling language. Morgan Kaufmann"},{"key":"10353_CR22","doi-asserted-by":"publisher","unstructured":"Garavel H, ter Beek MH, van\u00a0de Pol J (2020) The 2020 expert survey on formal methods. In: Formal Methods for Industrial Critical Systems - 25th International Conference, FMICS 2020, Vienna, Austria, September 2-3, 2020, Proceedings, pp 3\u201369. https:\/\/doi.org\/10.1007\/978-3-030-58298-2_1","DOI":"10.1007\/978-3-030-58298-2_1"},{"key":"10353_CR23","unstructured":"Gerking C, Sch\u00e4fer W, Dziwok S, Heinzemann C (2015) Domain-specific model checking for cyber-physical systems. In: Famelis M, Ratiu D, Seidl M, Selim GMK (eds) Proceedings of the 12th Workshop on Model-Driven Engineering, Verification and Validation co-located with ACM\/IEEE 18th International Conference on Model Driven Engineering Languages and Systems, MoDeVVa@MoDELS 2015, Ottawa, Canada, September 29, 2015, vol 1514 of CEUR Workshop Proceedings. pp 18\u201327. CEUR-WS.org. http:\/\/ceur-ws.org\/Vol-1514\/paper3.pdf"},{"key":"10353_CR24","doi-asserted-by":"publisher","unstructured":"Giannakopoulou D, Pressburger T, Mavridou A, Schumann J (2020) Generation of formal requirements from structured natural language. In: Madhavji NH, Pasquale L, Ferrari A, Gnesi S (eds) Requirements Engineering: Foundation for Software Quality - 26th International Working Conference, REFSQ 2020, Pisa, Italy, March 24-27, 2020, Proceedings [REFSQ 2020 was postponed], volume 12045 of Lecture Notes in Computer Science. Springer, pp 19\u201335. https:\/\/doi.org\/10.1007\/978-3-030-44429-7_2","DOI":"10.1007\/978-3-030-44429-7_2"},{"key":"10353_CR25","doi-asserted-by":"publisher","unstructured":"Gladisch C, Heinz T, Heinzemann C, Oehlerking J, von Vietinghoff A, Pfitzer T (2019) Experience paper: Search-based testing in automated driving control applications. In: 34th IEEE\/ACM International Conference on Automated Software Engineering, ASE 2019, San Diego, CA, USA, November 11-15, 2019. pp 26\u201337. https:\/\/doi.org\/10.1109\/ASE.2019.00013","DOI":"10.1109\/ASE.2019.00013"},{"issue":"6","key":"10353_CR26","doi-asserted-by":"publisher","first-page":"4473","DOI":"10.1007\/s10664-020-09836-5","volume":"25","author":"M Gleirscher","year":"2020","unstructured":"Gleirscher M, Marmsoler D (2020) Formal methods in dependable systems engineering: a survey of professionals from europe and north america. Empir Softw Eng 25(6):4473\u20134546. https:\/\/doi.org\/10.1007\/s10664-020-09836-5","journal-title":"Empir Softw Eng"},{"key":"10353_CR27","doi-asserted-by":"publisher","unstructured":"Grumberg O, Veith H (eds) 25 Years of Model Checking - History, Achievements, Perspectives, vol 5000 of Lecture Notes in Computer Science. Springer. ISBN 978-3-540-69849-4. https:\/\/doi.org\/10.1007\/978-3-540-69850-0","DOI":"10.1007\/978-3-540-69850-0"},{"key":"10353_CR28","doi-asserted-by":"publisher","unstructured":"Grunske L (2008) Specification patterns for probabilistic quality properties. In: Sch\u00e4fer W, Dwyer MB, Gruhn V (eds) 30th International Conference on Software Engineering (ICSE 2008), Leipzig, Germany, May 10-18, 2008, pages 31\u201340. ACM, 2008. https:\/\/doi.org\/10.1145\/1368088.1368094","DOI":"10.1145\/1368088.1368094"},{"key":"10353_CR29","doi-asserted-by":"publisher","unstructured":"Heitmeyer CL (1998) On the need for practical formal methods. In: Ravn AP, Rischel H (eds) Formal Techniques in Real-Time and Fault-Tolerant Systems, 5th International Symposium, FTRTFT\u201998, Lyngby, Denmark, September 14-18, 1998, Proceedings, volume 1486 of Lecture Notes in Computer Science. Springer, pp 18\u201326. https:\/\/doi.org\/10.1007\/BFb0055332","DOI":"10.1007\/BFb0055332"},{"issue":"1","key":"10353_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3522577","volume":"34","author":"CB Jones","year":"2022","unstructured":"Jones CB, Thomas M (2022) The development and deployment of formal methods in the UK. Formal Aspects Comput 34(1):1\u201321. https:\/\/doi.org\/10.1145\/3522577","journal-title":"Formal Aspects Comput"},{"key":"10353_CR31","doi-asserted-by":"publisher","first-page":"66","DOI":"10.7250\/csimq.2015-4.05","volume":"4","author":"B Kaiser","year":"2015","unstructured":"Kaiser B, Weber R, Oertel M, B\u00f6de E, Nejad BM, Zander J (2015) Contract-based design of embedded systems integrating nominal behavior and safety. Complex Syst Informatics Model Q 4:66\u201391. https:\/\/doi.org\/10.7250\/csimq.2015-4.05","journal-title":"Complex Syst Informatics Model Q"},{"key":"10353_CR32","doi-asserted-by":"crossref","unstructured":"Kaleeswaran AP, Nordmann A, Vogel T, Grunske L (2020) Counterexample interpretation for contract-based design. In: Model-Based Safety and Assessment - 7th International Symposium, IMBSA 2020, Lisbon, Portugal, September 14-16, 2020, Proceedings. pp 99\u2013114","DOI":"10.1007\/978-3-030-58920-2_7"},{"key":"10353_CR33","doi-asserted-by":"crossref","unstructured":"Kaleeswaran AP, Nordmann A, Vogel T, Grunske L (2021) A user-study protocol for evaluation of formal verification results and their explanation. CoRR. arXiv:2108.06376","DOI":"10.26226\/morressier.613b5418842293c031b5b5f1"},{"key":"10353_CR34","doi-asserted-by":"publisher","unstructured":"Kaleeswaran AP, Nordmann A, Vogel T, Grunske L (2022) A systematic literature review on counterexample explanation. Information and Software Technology 145:106800. ISSN 0950-5849. https:\/\/doi.org\/10.1016\/j.infsof.2021.106800","DOI":"10.1016\/j.infsof.2021.106800"},{"key":"10353_CR35","doi-asserted-by":"publisher","unstructured":"Khazeev M, Aslam H, de\u00a0Carvalho D, Mazzara M, Bruel J, Brown JA (2019) Reflections on teaching formal methods for software development in higher education. In: Bruel J, Capozucca A, Mazzara M, Meyer B, Naumchev A, Sadovykh A (eds) Frontiers in Software Engineering Education - First International Workshop, FISEE 2019, Villebrumier, France, November 11-13, 2019, Invited Papers, volume 12271 of Lecture Notes in Computer Science. Springer, pp 28\u201341. https:\/\/doi.org\/10.1007\/978-3-030-57663-9_3","DOI":"10.1007\/978-3-030-57663-9_3"},{"key":"10353_CR36","doi-asserted-by":"publisher","unstructured":"Kitchenham BA, Pfleeger SL (2008) Personal opinion surveys. In: Guide to Advanced Empirical Software Engineering. pp 63\u201392. https:\/\/doi.org\/10.1007\/978-1-84800-044-5_3","DOI":"10.1007\/978-1-84800-044-5_3"},{"key":"10353_CR37","doi-asserted-by":"publisher","unstructured":"Konrad S, Cheng BHC (2005) Real-time specification patterns. In: 27th International Conference on Software Engineering (ICSE 2005), 15-21 May 2005, St. Louis, Missouri, USA. pp 372\u2013381. https:\/\/doi.org\/10.1145\/1062455.1062526","DOI":"10.1145\/1062455.1062526"},{"key":"10353_CR38","doi-asserted-by":"publisher","unstructured":"Kossak F, Mashkoor A, Geist V, Illibauer C (2014) Improving the understandability of formal specifications: An experience report. In: Salinesi C, van\u00a0de Weerd I (eds) Requirements Engineering: Foundation for Software Quality - 20th International Working Conference, REFSQ 2014, Essen, Germany, April 7-10, 2014. Proceedings, volume 8396 of Lecture Notes in Computer Science. Springer, pp 184\u2013199. https:\/\/doi.org\/10.1007\/978-3-319-05843-6_14","DOI":"10.1007\/978-3-319-05843-6_14"},{"key":"10353_CR39","unstructured":"McMillan KL (1999) The smv language. Cadence Berkeley Labs. pp 1\u201349"},{"key":"10353_CR40","unstructured":"Neuman WL (2014) Basics of social research. Pearson\/Allyn and Bacon"},{"key":"10353_CR41","doi-asserted-by":"publisher","unstructured":"Paulson LC (1994) Isabelle - A Generic Theorem Prover (with a contribution by T. Nipkow), volume 828 of Lecture Notes in Computer Science. Springer, ISBN 3-540-58244-4. https:\/\/doi.org\/10.1007\/BFb0030541","DOI":"10.1007\/BFb0030541"},{"key":"10353_CR42","doi-asserted-by":"publisher","unstructured":"Pnueli A (1977) The temporal logic of programs. In: 18th Annual Symposium on Foundations of Computer Science, Providence, Rhode Island, USA, 31 October - 1 November 1977. IEEE Computer Society, pp 46\u201357. https:\/\/doi.org\/10.1109\/SFCS.1977.32","DOI":"10.1109\/SFCS.1977.32"},{"key":"10353_CR43","unstructured":"Pohl K, Rupp C (2011) Requirements Engineering Fundamentals - A Study Guide for the Certified Professional for Requirements Engineering Exam: Foundation Level - IREB compliant. rockynook, ISBN 978-1-933952-81-9"},{"key":"10353_CR44","doi-asserted-by":"crossref","unstructured":"Post A, Hoenicke J (2012) Formalization and analysis of real-time requirements: A feasibility study at BOSCH. In: Verified Software: Theories, Tools, Experiments - 4th International Conference, VSTTE 2012, Philadelphia, PA, USA, January 28-29, 2012. Proceedings, pp 225\u2013240","DOI":"10.1007\/978-3-642-27705-4_18"},{"issue":"1","key":"10353_CR45","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1007\/s00766-011-0145-9","volume":"17","author":"A Post","year":"2012","unstructured":"Post A, Menzel I, Hoenicke J, Podelski A (2012) Automotive behavioral requirements expressed in a specification pattern system: a case study at BOSCH. Requir Eng 17(1):19\u201333","journal-title":"Requir Eng"},{"key":"10353_CR46","doi-asserted-by":"publisher","unstructured":"Ratiu D, Nordmann A, Munk P, Carlan C, Voelter M (2021) FASTEN: An Extensible Platform to Experiment with Rigorous Modeling of Safety-Critical Systems. Springer International Publishing, pp 131\u2013164. ISBN 978-3-030-73758-0. https:\/\/doi.org\/10.1007\/978-3-030-73758-0_5","DOI":"10.1007\/978-3-030-73758-0_5"},{"key":"10353_CR47","unstructured":"Reid A, Church L, Flur S, de\u00a0Haas S, Johnson M, Laurie B (2020) Towards making formal methods normal: meeting developers where they are. CoRR. arXiv:2010.16345"},{"key":"10353_CR48","unstructured":"Robbins NB, Heiberger RM (2011) Plotting likert and other rating scales. In: Proceedings of the 2011 Joint Statistical Meeting vol 1"},{"key":"10353_CR49","unstructured":"Robson C, McCartan K (2016) Real world research. John Wiley & Sons, 2016"},{"key":"10353_CR50","doi-asserted-by":"publisher","unstructured":"Rodrigues P, Ecar M, Menezes SV, da\u00a0Silva JPS, Guedes GTA, Rodrigues EM (2018) Empirical evaluation of formal method for requirements specification in agile approaches. In: Boscarioli C, Costa CA, de\u00a0Avila\u00a0e Silva S, Notari DL (eds) Proceedings of the XIV Brazilian Symposium on Information Systems, SBSI 2018, Caxias do Sul, Brazil, June 04-08, 2018. ACM. pp 53:1\u201353:8. https:\/\/doi.org\/10.1145\/3229345.3229401","DOI":"10.1145\/3229345.3229401"},{"key":"10353_CR51","volume-title":"Formal methods and the certification of critical systems","author":"J Rushby","year":"1993","unstructured":"Rushby J (1993) Formal methods and the certification of critical systems, vol 37. SRI International, Computer Science Laboratory"},{"issue":"4","key":"10353_CR52","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1016\/S0950-5849(00)00166-X","volume":"43","author":"CF Snook","year":"2001","unstructured":"Snook CF, Harrison R (2001) Practitioners\u2019 views on the use of formal methods: an industrial survey by structured interview. Inf Softw Technol 43(4):275\u2013283. https:\/\/doi.org\/10.1016\/S0950-5849(00)00166-X","journal-title":"Inf Softw Technol"},{"key":"10353_CR53","doi-asserted-by":"publisher","unstructured":"ter Beek MH, Bor\u00e4lv A, Fantechi A, Ferrari A, Gnesi S, L\u00f6fving C, Mazzanti F (2019) Adopting formal methods in an industrial setting: The railways case. In: ter Beek MH, McIver A, Oliveira JN (eds) Formal Methods - The Next 30 Years - Third World Congress, FM 2019, Porto, Portugal, October 7-11, 2019, Proceedings, volume 11800 of Lecture Notes in Computer Science. Springer, pp 762\u2013772. https:\/\/doi.org\/10.1007\/978-3-030-30942-8_46","DOI":"10.1007\/978-3-030-30942-8_46"},{"key":"10353_CR54","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-01253-2","author":"J Weber","year":"2009","unstructured":"Weber J (2009) Automotive Development Processes: Processes for Successful Customer Oriented Vehicle Development. Springer, Berlin Heidelberg. https:\/\/doi.org\/10.1007\/978-3-642-01253-2","journal-title":"Springer, Berlin Heidelberg."},{"issue":"9","key":"10353_CR55","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1109\/2.58215","volume":"23","author":"JM Wing","year":"1990","unstructured":"Wing JM (1990) A specifier\u2019s introduction to formal methods. Computer 23(9):8\u201324. https:\/\/doi.org\/10.1109\/2.58215","journal-title":"Computer"},{"key":"10353_CR56","doi-asserted-by":"publisher","unstructured":"Wohlin C, Runeson P, H\u00f6st M, Ohlsson MC, Regnell B (2012) Experimentation in Software Engineering. Springer. ISBN 978-3-642-29043-5.https:\/\/doi.org\/10.1007\/978-3-642-29044-2","DOI":"10.1007\/978-3-642-29044-2"},{"issue":"2","key":"10353_CR57","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1007\/s10664-012-9200-5","volume":"18","author":"A Zaidman","year":"2013","unstructured":"Zaidman A, Matthijssen N, Storey MD, van Deursen A (2013) Understanding ajax applications by connecting client and server-side execution traces. Empir Softw Eng 18(2):181\u2013218. https:\/\/doi.org\/10.1007\/s10664-012-9200-5","journal-title":"Empir Softw Eng"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10353-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-023-10353-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10353-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T12:11:40Z","timestamp":1711023100000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-023-10353-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9]]},"references-count":57,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2023,9]]}},"alternative-id":["10353"],"URL":"https:\/\/doi.org\/10.1007\/s10664-023-10353-4","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9]]},"assertion":[{"value":"30 May 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 September 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of Interest"}}],"article-number":"125"}}