{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T21:00:35Z","timestamp":1774731635904,"version":"3.50.1"},"reference-count":129,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,2,26]],"date-time":"2024-02-26T00:00:00Z","timestamp":1708905600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,2,26]],"date-time":"2024-02-26T00:00:00Z","timestamp":1708905600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"TU Wien"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2024,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Context<\/jats:title>\n                <jats:p>The readability of source code is key for understanding and maintaining software systems and tests. Although several studies investigate the readability of source code, there is limited research specifically on the readability of test code and related influence factors.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Objective<\/jats:title>\n                <jats:p>In this paper, we aim at investigating the factors that influence the readability of test code from an academic perspective based on scientific literature sources and complemented by practical views, as discussed in grey literature.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>First, we perform a Systematic Mapping Study (SMS) with a focus on scientific literature. Second, we extend this study by reviewing grey literature sources for practical aspects on test code readability and understandability. Finally, we conduct a controlled experiment on the readability of a selected set of test cases to collect additional knowledge on influence factors discussed in practice.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The result set of the SMS includes 19 primary studies from the scientific literature for further analysis. The grey literature search reveals 62 sources for information on test code readability. Based on an analysis of these sources, we identified a combined set of 14 factors that influence the readability of test code. 7 of these factors were found in scientific <jats:italic>and<\/jats:italic> grey literature, while some factors were mainly discussed in academia (2) <jats:italic>or<\/jats:italic> industry (5) with only limited overlap. The controlled experiment on practically relevant influence factors showed that the investigated factors have a significant impact on readability for half of the selected test cases.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>Our review of scientific and grey literature showed that test code readability is of interest for academia and industry with a consensus on key influence factors. However, we also found factors only discussed by practitioners. For some of these factors we were able to confirm an impact on readability in a first experiment. Therefore, we see the need to bring together academic and industry viewpoints to achieve a common view on the readability of software test code.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s10664-023-10390-z","type":"journal-article","created":{"date-parts":[[2024,2,26]],"date-time":"2024-02-26T12:02:45Z","timestamp":1708948965000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Investigating the readability of test code"],"prefix":"10.1007","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4743-3124","authenticated-orcid":false,"given":"Dietmar","family":"Winkler","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pirmin","family":"Urbanke","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rudolf","family":"Ramler","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,2,26]]},"reference":[{"key":"10390_CR1","doi-asserted-by":"crossref","unstructured":"Afshan S, McMinn P, Stevenson M (2013) Evolving readable string test inputs using a natural language model to reduce human oracle cost. In: Proceedings - IEEE 6th Int. Conf. on Software Testing, Verification and Validation, ICST 2013, pp 352\u2013361","DOI":"10.1109\/ICST.2013.11"},{"key":"10390_CR2","doi-asserted-by":"crossref","unstructured":"Al\u00a0Madi N (2022) How readable is model-generated code? examining readability and visual inspection of github copilot. In: 37th IEEE\/ACM International conference on automated software engineering, pp 1\u20135","DOI":"10.1145\/3551349.3560438"},{"key":"10390_CR3","doi-asserted-by":"crossref","unstructured":"Almasi M, Hemmati H, Fraser G, Arcuri A, Benefelds J (2017) An industrial evaluation of unit test generation: Finding real faults in a financial application. In: Proceedings - 2017 IEEE\/ACM 39th Int. Conf. on Software Engineering: Software Engineering in Practice Track, ICSE-SEIP 2017, Institute of Electrical and Electronics Engineers Inc., pp 263\u2013272","DOI":"10.1109\/ICSE-SEIP.2017.27"},{"key":"10390_CR4","doi-asserted-by":"publisher","unstructured":"Alsharif A, Kapfhammer GM, McMinn P (2019) What factors make sql test cases understandable for testers? a human study of automated test data generation techniques. In: 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp 437\u2013448, https:\/\/doi.org\/10.1109\/ICSME.2019.00076","DOI":"10.1109\/ICSME.2019.00076"},{"key":"10390_CR5","unstructured":"Aniche M (2022) Effective Software Testing Version 4 - Academia.edu. nan, https:\/\/www.academia.edu\/70086528\/Effective_Software_Testing_Version_4"},{"key":"10390_CR6","unstructured":"Bahmutov G (2019) Readable cypress.io tests - gleb bahmutov. Blog, https:\/\/glebbahmutov.com\/blog\/readable-tests\/"},{"key":"10390_CR7","doi-asserted-by":"publisher","unstructured":"Bai GR, Presler-Marshall K, Fisk SR, Stolee KT (2022) Is assertion roulette still a test smell? an experiment from the perspective of testing education. In: 2022 IEEE Symposium on visual languages and human-centric computing (VL\/HCC), pp 1\u20137, https:\/\/doi.org\/10.1109\/VL\/HCC53370.2022.9833107","DOI":"10.1109\/VL\/HCC53370.2022.9833107"},{"key":"10390_CR8","unstructured":"Bansal A (2021) Best practices for unit testing in java - baeldung. Magazine, https:\/\/www.baeldung.com\/java-unit-testing-best-practices"},{"issue":"4","key":"10390_CR9","doi-asserted-by":"publisher","first-page":"1052","DOI":"10.1007\/s10664-014-9313-0","volume":"20","author":"G Bavota","year":"2015","unstructured":"Bavota G, Qusef A, Oliveto R, De Lucia A, Binkley D (2015) Are test smells really harmful? an empirical study. Empirical Software Engineering 20(4):1052\u20131094","journal-title":"Empirical Software Engineering"},{"key":"10390_CR10","unstructured":"blog VVC (2021) The unit test strategy in vald. Blog, https:\/\/vdaas-vald.medium.com\/the-unit-test-strategy-in-vald-912ed6f14fbd"},{"key":"10390_CR11","unstructured":"Bloomfield P (2020) How to write a good unit test - peter bloomfield. Blog, https:\/\/peter.bloomfield.online\/how-to-write-a-good-unit-test\/"},{"key":"10390_CR12","unstructured":"Borowiec R (2013) Test code readability improved: Junit with mockito and fest ... Blog, https:\/\/blog.codeleak.pl\/2013\/07\/test-code-readability-improved-junit.html"},{"key":"10390_CR13","first-page":"9","volume-title":"How good are my tests?","author":"D Bowes","year":"2017","unstructured":"Bowes D, Hall T, Petri\u0107 J, Shippey T, Turhan B (2017) How good are my tests? IEEE Computer Society, Workshop on Emerging Trends in Software Metrics, Int. WETSoM, pp 9\u201314"},{"key":"10390_CR14","unstructured":"Brizeno M (2014) Write better tests in 5 steps - thoughtworks. Blog, https:\/\/www.thoughtworks.com\/insights\/blog\/write-better-tests-5-steps"},{"key":"10390_CR15","unstructured":"Bulavin V (2020) Vadim bulavin auf twitter: \"9. a good unit test must have three ... Blog, https:\/\/twitter.com\/v8tr\/status\/1217483476406079491?lang=de"},{"key":"10390_CR16","doi-asserted-by":"crossref","unstructured":"Buse RP, Weimer WR (2008) A metric for software readability. In: Proceedings of the 2008 int. symposium on Software testing and analysis, pp 121\u2013130","DOI":"10.1145\/1390630.1390647"},{"key":"10390_CR17","unstructured":"Carter D (2011) Make your automated tests easy to read - dc coding - dan ... Blog, https:\/\/codingblog.carterdan.net\/2020\/02\/11\/make-your-automated-tests-easy-to-read\/"},{"key":"10390_CR18","unstructured":"Casadei R (2013) Effective unit testing - slideshare. Presentation, https:\/\/de.slideshare.net\/RobertoCasadei\/effective-unit-testing"},{"key":"10390_CR19","doi-asserted-by":"publisher","unstructured":"Ceccato M, Marchetto A, Mariani L, Nguyen CD, Tonella P (2015) Do automatically generated test cases make debugging easier? an experimental assessment of debugging effectiveness and efficiency https:\/\/doi.org\/10.1145\/2768829","DOI":"10.1145\/2768829"},{"key":"10390_CR20","unstructured":"Coles H, et\u00a0al. (2015) Write damp test code - java for small teams - ncrcoe. Wiki, https:\/\/ncrcoe.gitbooks.io\/java-for-small-teams\/content\/v\/restructure\/tests\/1900_write_damp_test_code.html"},{"key":"10390_CR21","unstructured":"Cook R (2017) In programming, is it worth to sacrifice some dry-ness for ... Other, https:\/\/www.quora.com\/In-Programming-is-it-worth-to-sacrifice-some-DRY-ness-for-code-readability-or-the-other-way-around"},{"key":"10390_CR22","unstructured":"Correa FC (2019) How to write legible qa tests - codacy - blog. Blog, https:\/\/blog.codacy.com\/how-to-write-legible-qa-tests\/"},{"key":"10390_CR23","doi-asserted-by":"crossref","unstructured":"Daka E, Campos J, Fraser G, Dorn J, Weimer W (2015) Modeling readability to improve unit tests. In: 2015 10th Joint Meeting of the European Software Engineering Conf. and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC\/FSE 2015 - Proceedings, Association for Computing Machinery, Inc, pp 107\u2013118","DOI":"10.1145\/2786805.2786838"},{"key":"10390_CR24","doi-asserted-by":"crossref","unstructured":"Daka E, Rojas J, Fraser G (2017) Generating unit tests with descriptive names or: Would you name your children thing1 and thing2? In: ISSTA 2017 - Proceedings of the 26th ACM SIGSOFT Int. Symposium on Software Testing and Analysis, Association for Computing Machinery, Inc, pp 57\u201367","DOI":"10.1145\/3092703.3092727"},{"issue":"4","key":"10390_CR25","doi-asserted-by":"publisher","first-page":"347","DOI":"10.1007\/s10009-008-0079-9","volume":"10","author":"T Dei\u00df","year":"2008","unstructured":"Dei\u00df T (2008) Refactoring and converting a ttcn-2 test suite. Int Journal on Software Tools for Technology Transfer 10(4):347\u2013352","journal-title":"Int Journal on Software Tools for Technology Transfer"},{"key":"10390_CR26","doi-asserted-by":"crossref","unstructured":"Deissenboeck F, Wagner S, Pizka M, Teuchert S, Girard JF (2007) An activity-based quality model for maintainability. In: 2007 IEEE International conference on software maintenance, IEEE, pp 184\u2013193","DOI":"10.1109\/ICSM.2007.4362631"},{"key":"10390_CR27","doi-asserted-by":"crossref","unstructured":"Delplanque J, Ducasse S, Polito G, Black AP, Etien A (2019) Rotten green tests. In: 2019 IEEE\/ACM 41st International conference on software engineering (ICSE), IEEE, pp 500\u2013511","DOI":"10.1109\/ICSE.2019.00062"},{"key":"10390_CR28","unstructured":"Dhoke A (2009) Do you really want moist test - ameydhoke\u2019s blog. Blog, http:\/\/maverick-amey.blogspot.com\/2009\/05\/do-you-really-want-moist-test.html"},{"key":"10390_CR29","unstructured":"Dietrich E (2013) Test readability: Best of all worlds - daedtech. Blog, https:\/\/daedtech.com\/test-readability-best-of-all-worlds\/"},{"key":"10390_CR30","unstructured":"Dijkstra B (2016) Three practices for creating readable test code. Blog, https:\/\/www.ontestautomation.com\/three-practices-for-creating-readable-test-code\/"},{"key":"10390_CR31","unstructured":"Duiker M (2016) Improving unit test readability: helper methods & named ... Blog, https:\/\/blog.marcduiker.nl\/2016\/06\/01\/improving-unit-test-readability-named-args.html"},{"key":"10390_CR32","unstructured":"Dustin\u00a0Boswell TF (2012) The Art of Readable Code. O\u2019Reilly, https:\/\/www.oreilly.com\/library\/view\/the-art-of\/9781449318482\/ch14.html"},{"key":"10390_CR33","unstructured":"Enzler U (2014) Clean code cheat sheet - planetgeek.ch. Cheatsheet, https:\/\/www.planetgeek.ch\/wp-content\/uploads\/2014\/11\/Clean-Code-V2.4.pdf"},{"key":"10390_CR34","doi-asserted-by":"crossref","unstructured":"Femmer H, Mund J, Fern\u00e1ndez DM (2015) It\u2019s the activities, stupid! a new perspective on re quality. In: 2015 IEEE\/ACM 2nd International workshop on requirements engineering and testing, IEEE, pp 13\u201319","DOI":"10.1109\/RET.2015.11"},{"key":"10390_CR35","unstructured":"Fernandes J (2020) Rethinking testing through declarative programming. Blog, https:\/\/betterprogramming.pub\/rethinking-testing-through-declarative-programming-335897703bdd"},{"key":"10390_CR36","doi-asserted-by":"crossref","unstructured":"Fisher G, Johnson C (2018) Specification-based testing in software engineering courses. In: SIGCSE 2018 - Proc. of the 49th ACM Techn. Symposium on Computer Science Education, Association for Computing Machinery, Inc, vol 2018-January, pp 800\u2013805","DOI":"10.1145\/3159450.3159575"},{"key":"10390_CR37","unstructured":"Flournoy J (2008) Category: coverage - pervasive code. Blog, http:\/\/www.pervasivecode.com\/blog\/category\/coverage\/"},{"key":"10390_CR38","unstructured":"Foord M (2017) 30 best practices for software development and testing. Magazine, https:\/\/opensource.com\/article\/17\/5\/30-best-practices-software-development-and-testing"},{"key":"10390_CR39","doi-asserted-by":"publisher","unstructured":"Fraser G, Staats M, McMinn P, Arcuri A, Padberg F (2013) Does automated white-box test generation really help software testers? In: Proceedings of the 2013 International symposium on software testing and analysis, Association for Computing Machinery, ISSTA 2013, p 291-301, https:\/\/doi.org\/10.1145\/2483760.2483774","DOI":"10.1145\/2483760.2483774"},{"key":"10390_CR40","doi-asserted-by":"crossref","unstructured":"Fraser G, Zeller A (2011) Exploiting common object usage in test case generation. In: Proceedings - 4th IEEE Int. Conf. on Software Testing, Verification, and Validation, ICST 2011, pp 80\u201389","DOI":"10.1109\/ICST.2011.53"},{"issue":"3","key":"10390_CR41","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1109\/MS.2016.30","volume":"33","author":"V Garousi","year":"2016","unstructured":"Garousi V, Felderer M (2016) Developing, verifying, and maintaining high-quality automated test scripts. IEEE Software 33(3):68\u201375","journal-title":"IEEE Software"},{"key":"10390_CR42","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1016\/j.jss.2017.12.013","volume":"138","author":"V Garousi","year":"2018","unstructured":"Garousi V, K\u00fc\u00e7\u00fck B (2018) Smells in software test code: a survey of knowledge in industry and academia. Journal of Systems and Software 138:52\u201381","journal-title":"Journal of Systems and Software"},{"key":"10390_CR43","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1016\/j.infsof.2018.09.006","volume":"106","author":"V Garousi","year":"2019","unstructured":"Garousi V, Felderer M, M\u00e4ntyl\u00e4 MV (2019) Guidelines for including grey literature and conducting multivocal literature reviews in software engineering. Information and Software Technology 106:101\u2013121","journal-title":"Information and Software Technology"},{"key":"10390_CR44","unstructured":"Goeschel T (2016) Writing better tests with junit - codecentric ag blog. Blog, https:\/\/blog.codecentric.de\/en\/2016\/01\/writing-better-tests-junit\/"},{"key":"10390_CR45","unstructured":"Gorman J (2020) Readable parameterized tests - codemanship\u2019s blog. Blog, https:\/\/codemanship.wordpress.com\/2020\/09\/26\/readable-parameterized-tests\/"},{"key":"10390_CR46","doi-asserted-by":"crossref","unstructured":"Grano G, De\u00a0Iaco C, Palomba F, Gall HC (2020) Pizza versus pinsa: On the perception and measurability of unit test code quality. In: 2020 IEEE Int. conf. on software maintenance and evolution (ICSME), IEEE, pp 336\u2013347","DOI":"10.1109\/ICSME46990.2020.00040"},{"key":"10390_CR47","doi-asserted-by":"crossref","unstructured":"Grano G, Scalabrino S, Gall H, Oliveto R (2018a) An empirical investigation on the readability of manual and generated test cases. In: Proceedings of the int. conf. on software engineering, IEEE computer society, pp 348\u2013351","DOI":"10.1145\/3196321.3196363"},{"key":"10390_CR48","doi-asserted-by":"crossref","unstructured":"Grano G, Scalabrino S, Gall H, Oliveto R (2018b) An empirical investigation on the readability of manual and generated test cases. In: Proceedings of the Int. Conf. on Software Engineering, IEEE Computer Society, pp 348\u2013351","DOI":"10.1145\/3196321.3196363"},{"key":"10390_CR49","unstructured":"Grigg H (2020) A simple, readable, meaningful test style with jest. Blog, https:\/\/notestoself.dev\/posts\/simple-readable-meaningful-jest-test-style\/"},{"key":"10390_CR50","unstructured":"Hauer P (2021) Modern best practices for testing in java - philipp hauer\u2019s blog. Blog, https:\/\/phauer.com\/2019\/modern-best-practices-testing-java\/"},{"key":"10390_CR51","unstructured":"Haus E (2020) 102. whether or not to repeat yourself: Dry, damp, or wet. Podcast \/ transcript, https:\/\/www.heroku.com\/podcasts\/codeish\/102-whether-to-repeat-yourself-dry-damp-or-wet"},{"key":"10390_CR52","unstructured":"Hnat B (2020) Dryer tests - the dumpster fire project. Blog, https:\/\/thedumpsterfireproject.com\/dryer-tests"},{"key":"10390_CR53","unstructured":"Huttunen A (2021) How to make your tests readable - arho huttunen. Blog, https:\/\/www.arhohuttunen.com\/test-readability\/"},{"key":"10390_CR54","unstructured":"Jarrett J (2009) Fluent specification extensions - developing on staxmanade. Blog, https:\/\/staxmanade.com\/2009\/02\/fluent-specification-extensions\/"},{"key":"10390_CR55","unstructured":"Johnson K (2008) Is duplicated code more tolerable in unit tests? - stack overflow. Stackoverflow, https:\/\/stackoverflow.com\/questions\/129693\/is-duplicated-code-more-tolerable-in-unit-tests"},{"key":"10390_CR56","unstructured":"Kainulainen P (2014) Writing clean tests - petri kainulainen. Blog, https:\/\/www.petrikainulainen.net\/writing-clean-tests\/"},{"key":"10390_CR57","unstructured":"Kareinen T (2012) Readable tests - tuomas kareinen\u2019s blog. Blog, https:\/\/tkareine.org\/articles\/readable-tests.html"},{"key":"10390_CR58","unstructured":"Khorikov V (2008) Dry vs damp in unit tests - enterprise craftsmanship. Blog, https:\/\/enterprisecraftsmanship.com\/posts\/dry-damp-unit-tests\/"},{"key":"10390_CR59","doi-asserted-by":"crossref","unstructured":"Kochhar PS, Xia X, Lo D (2019) Practitioners\u2019 views on good software testing practices. 2019 IEEE\/ACM 41st Int. conf. on software engineering, software engineering in practice (ICSE-SEIP), IEEE, pp 61\u201370","DOI":"10.1109\/ICSE-SEIP.2019.00015"},{"key":"10390_CR60","unstructured":"Koskela L (2013) Effective Unit Testing. Manning, https:\/\/livebook.manning.com\/effective-unit-testing\/chapter-4"},{"key":"10390_CR61","doi-asserted-by":"publisher","first-page":"110610","DOI":"10.1016\/j.jss.2020.110610","volume":"167","author":"G Lacerda","year":"2020","unstructured":"Lacerda G, Petrillo F, Pimenta M, Gu\u00e9h\u00e9neuc YG (2020) Code smells and refactoring: a tertiary systematic review of challenges and observations. Journal of Systems and Software 167:110610","journal-title":"Journal of Systems and Software"},{"key":"10390_CR62","unstructured":"Lal A (2019) Kotlin dsl - let\u2019s express code in \"mini-language\" - part 5 of 5. Blog, https:\/\/www.aditlal.dev\/kotlin-dsl-part-5\/"},{"issue":"3","key":"10390_CR63","doi-asserted-by":"publisher","first-page":"753","DOI":"10.1007\/s10664-013-9281-9","volume":"19","author":"R Latorre","year":"2014","unstructured":"Latorre R (2014) A successful application of a test-driven development strategy in the industrial environment. Empirical Software Engineering 19(3):753\u2013773","journal-title":"Empirical Software Engineering"},{"key":"10390_CR64","unstructured":"Lehner D (2021) 3 easy fixes for perfect unit test code - devmate. Blog, https:\/\/www.devmate.software\/3-easy-fixes-for-perfect-unit-test-code\/"},{"key":"10390_CR65","doi-asserted-by":"publisher","unstructured":"Leotta M, Cerioli M, Olianas D, Ricca F (2018a) Fluent vs basic assertions in java: an empirical study. In: 2018 11th International conference on the quality of information and communications technology (QUATIC), pp 184\u2013192, https:\/\/doi.org\/10.1109\/QUATIC.2018.00036","DOI":"10.1109\/QUATIC.2018.00036"},{"key":"10390_CR66","doi-asserted-by":"publisher","unstructured":"Leotta M, Cerioli M, Olianas D, Ricca F (2018b) Fluent vs basic assertions in java: An empirical study. In: 2018 11th International Conference on the Quality of Information and Communications Technology (QUATIC), pp 184\u2013192, https:\/\/doi.org\/10.1109\/QUATIC.2018.00036","DOI":"10.1109\/QUATIC.2018.00036"},{"key":"10390_CR67","unstructured":"Lindner D (2013) unit test - schneide blog. Blog, https:\/\/schneide.blog\/tag\/unit-test\/"},{"key":"10390_CR68","doi-asserted-by":"crossref","unstructured":"Lin B, Nagy C, Bavota G, Marcus A, Lanza M (2019) On the quality of identifiers in test code. In: Proceedings - 19th IEEE int. working conf. on source code analysis and manipulation, SCAM 2019, Institute of Electrical and Electronics Engineers Inc., pp 204\u2013215","DOI":"10.1109\/SCAM.2019.00031"},{"key":"10390_CR69","unstructured":"Lipinski P (2013) or how to write tests so that they serve you well. Presentation, https:\/\/2013.jokerconf.com\/presentations\/03_02_lipinski_pawel_jokerconf-presentation.pdf"},{"key":"10390_CR70","doi-asserted-by":"publisher","unstructured":"Li B, Vendome C, Linares-V\u00e1squez M, Poshyvanyk D (2018) Aiding comprehension of unit test cases and test suites with stereotype-based tagging. In: Proceedings of the 26th conference on program comprehension, p 52\u201363, https:\/\/doi.org\/10.1145\/3196321.3196339","DOI":"10.1145\/3196321.3196339"},{"key":"10390_CR71","doi-asserted-by":"crossref","unstructured":"Li B, Vendome C, Linares-Vasquez M, Poshyvanyk D, Kraft N (2016) Automatically documenting unit test cases. In: Proceedings - 2016 IEEE Int. Conf. on software testing, verification and validation, ICST 2016, Institute of Electrical and Electronics Engineers Inc., pp 341\u2013352","DOI":"10.1109\/ICST.2016.30"},{"key":"10390_CR72","unstructured":"MANJUNATH N, MEULDER OD (2019) No code? no problem - writing tests in plain english - nyt ... Blog, https:\/\/open.nytimes.com\/no-code-no-problem-writing-tests-in-plain-english-537827eaaa6e"},{"key":"10390_CR73","unstructured":"Martin RC (2021) Clean Code: Chapter 9. nan, https:\/\/reee3.home.blog\/2021\/02\/17\/clean-code-9\/"},{"key":"10390_CR74","doi-asserted-by":"publisher","unstructured":"McMinn P, Shahbaz M, Stevenson M (2012) Search-based test input generation for string data types using the results of web queries. In: 2012 IEEE Fifth international conference on software testing, verification and validation, pp 141\u2013150, https:\/\/doi.org\/10.1109\/ICST.2012.94","DOI":"10.1109\/ICST.2012.94"},{"key":"10390_CR75","unstructured":"Meszaros G (2007) xUnit test patterns: Refactoring test code. Pearson Education"},{"key":"10390_CR76","doi-asserted-by":"crossref","unstructured":"Minelli R, Mocci A, Lanza M, (2015) I know what you did last summer-an investigation of how developers spend their time. (2015) IEEE 23rd Int. conf. on program comprehension, IEEE, pp 25\u201335","DOI":"10.1109\/ICPC.2015.12"},{"key":"10390_CR77","doi-asserted-by":"crossref","unstructured":"Moonen L, van Deursen A, Zaidman A, Bruntink M (2008) On the interplay between software testing and evolution and its effect on program comprehension. In: Software evolution, Springer, pp 173\u2013202","DOI":"10.1007\/978-3-540-76440-3_8"},{"key":"10390_CR78","unstructured":"Myers B (2021) Readable test code matters. - brooklin myers. Blog, https:\/\/brooklinmyers.medium.com\/readable-test-code-matters-e46cc5c411bb"},{"key":"10390_CR79","unstructured":"Needham M (2009) Tdd: Test dryness - mark needham. Blog, https:\/\/www.markhneedham.com\/blog\/2009\/01\/30\/tdd-test-dryness\/"},{"key":"10390_CR80","unstructured":"nickname Company\u00a0blog F (2016) More readable tests with kotlin - tengio. Blog, https:\/\/www.tengio.com\/blog\/more-readable-tests-with-kotlin\/"},{"key":"10390_CR81","doi-asserted-by":"crossref","unstructured":"Oliveira D, Bruno R, Madeiral F, Castor F (2020) Evaluating code readability and legibility: an examination of human-centric studies. In: 2020 IEEE Int. conf. on software maintenance and evolution (ICSME), IEEE, pp 348\u2013359","DOI":"10.1109\/ICSME46990.2020.00041"},{"key":"10390_CR82","doi-asserted-by":"crossref","unstructured":"Palomba F, Panichella A, Zaidman A, Oliveto R, De\u00a0Lucia A (2016) Automatic test case generation: What if test code quality matters? In: ISSTA 2016 - Proceedings of the 25th int. symposium on software testing and analysis, Association for Computing Machinery, Inc, pp 130\u2013141","DOI":"10.1145\/2931037.2931057"},{"issue":"7","key":"10390_CR83","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1007\/s10664-022-10207-5","volume":"27","author":"A Panichella","year":"2022","unstructured":"Panichella A, Panichella S, Fraser G, Sawant AA, Hellendoorn VJ (2022) Test smells 20 years later: detectability, validity, and reliability. Empirical Software Engineering 27(7):170","journal-title":"Empirical Software Engineering"},{"key":"10390_CR84","doi-asserted-by":"crossref","unstructured":"Panichella S, Panichella A, Beller M, Zaidman A, Gall H (2016) The impact of test case summaries on bug fixing performance: An empirical investigation. In: Proceedings - int. conf. on software engineering, IEEE computer society, vol 14-22-May-2016, pp 547\u2013558","DOI":"10.7287\/peerj.preprints.1467"},{"key":"10390_CR85","unstructured":"Papendieck T (2017) Why sometimes unit tests do more harm than good? Blog, https:\/\/www.beyondjava.net\/why-sometimes-unit-tests-do-more-harm-than-good"},{"key":"10390_CR86","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.infsof.2015.03.007","volume":"64","author":"K Petersen","year":"2015","unstructured":"Petersen K, Vakkalanka S, Kuzniarz L (2015) Guidelines for conducting systematic mapping studies in software engineering: an update. Information and Software Technology 64:1\u201318","journal-title":"Information and Software Technology"},{"key":"10390_CR87","unstructured":"Pip C (2020) Clean code in tests: What, why and how? - testproject. Blog, https:\/\/blog.testproject.io\/2020\/04\/22\/clean-code-in-tests-what-why-and-how\/"},{"key":"10390_CR88","doi-asserted-by":"crossref","unstructured":"Posnett D, Hindle A, Devanbu P (2011) A simpler model of software readability. In: Proceedings of the 8th working conf. on mining software repositories, pp 73\u201382","DOI":"10.1145\/1985441.1985454"},{"key":"10390_CR89","doi-asserted-by":"crossref","unstructured":"Ramler R, Klammer C, Buchgeher G, (2018) Applying automated test case generation in industry: a retrospective. (2018) IEEE Int. conf. on software testing, verification and validation workshops (ICSTW), IEEE, pp 364\u2013369","DOI":"10.1109\/ICSTW.2018.00074"},{"key":"10390_CR90","unstructured":"Reagan P (2009) Keep your friends close, but your test data closer - viget. Blog, https:\/\/www.viget.com\/articles\/keep-your-friends-close-but-your-test-data-closer\/"},{"key":"10390_CR91","unstructured":"Reid J (2016) 3 reasons why it\u2019s important to refactor tests - quality coding. Blog, https:\/\/qualitycoding.org\/why-refactor-tests\/"},{"issue":"2","key":"10390_CR92","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1016\/j.infsof.2008.01.007","volume":"51","author":"F Ricca","year":"2009","unstructured":"Ricca F, Torchiano M, Di Penta M, Ceccato M, Tonella P (2009) Using acceptance tests as a support for clarifying requirements: a series of experiments. Information and Software Technology 51(2):270\u2013283","journal-title":"Information and Software Technology"},{"key":"10390_CR93","unstructured":"Roberts J (2015) Improve test asserts with shouldly - visual studio magazine. Magazine, https:\/\/visualstudiomagazine.com\/articles\/2015\/08\/01\/improve-test-asserts-with-shouldly.aspx"},{"key":"10390_CR94","unstructured":"Roberts J (2019) Diagnosing failing tests more easily and improving test ... Blog, http:\/\/dontcodetired.com\/blog\/post\/Diagnosing-Failing-Tests-More-Easily-and-Improving-Test-Code-Readability"},{"key":"10390_CR95","unstructured":"Rodrigues M (2018) What makes good unit test? readability - matheus rodrigues. Blog, https:\/\/matheus.ro\/2018\/01\/15\/makes-good-unit-test-readability\/"},{"key":"10390_CR96","unstructured":"Romano J, Kromrey JD, Coraggio J, Skowronek J (2006) Appropriate statistics for ordinal level data: should we really be using t-test and cohen\u2019sd for evaluating group differences on the nsse and other surveys. In: annual meeting of the Florida Association of Institutional Research, vol 177, p\u00a034"},{"key":"10390_CR97","doi-asserted-by":"crossref","unstructured":"Roy D, Zhang Z, Ma M, Arnaoudova V, Panichella A, Panichella S, Gonzalez D, Mirakhorli M (2020a) Deeptc-enhancer: improving the readability of automatically generated tests. In: Proceedings - 2020 35th IEEE\/ACM Int. conf on automated software engineering, ASE 2020, Institute of Electrical and Electronics Engineers Inc., pp 287\u2013298","DOI":"10.1145\/3324884.3416622"},{"key":"10390_CR98","doi-asserted-by":"crossref","unstructured":"Roy D, Zhang Z, Ma M, Arnaoudova V, Panichella A, Panichella S, Gonzalez D, Mirakhorli M (2020b) Deeptc-enhancer: Improving the readability of automatically generated tests. In: Proceedings - 2020 35th IEEE\/ACM int. conf on automated software engineering, ASE 2020, Institute of Electrical and Electronics Engineers Inc., pp 287\u2013298","DOI":"10.1145\/3324884.3416622"},{"key":"10390_CR99","unstructured":"Ryswyck JV (2021) Avoid inheritance for test classes - principal it. Blog, https:\/\/principal-it.eu\/2021\/01\/avoid-inheritance-for-test-classes\/"},{"key":"10390_CR100","unstructured":"Sarna A (2018) Do you think your code is perfect? well, think again. Blog, https:\/\/blog.knoldus.com\/do-you-think-your-code-is-perfect-well-think-again\/"},{"key":"10390_CR101","unstructured":"Scalabrino S (2019) Automatically assessing and improving code readability and understandability. PhD thesis, Universit\u00e1 degli Studi del Molise, https:\/\/iris.unimol.it\/retrieve\/handle\/11695\/90885\/92359\/Tesi_S_Scalabrino.pdf"},{"key":"10390_CR102","doi-asserted-by":"crossref","unstructured":"Scalabrino S, Bavota G, Vendome C, Linares-V\u00e1squez M, Poshyvanyk D, Oliveto R (2017) Automatically assessing code understandability: how far are we? In: 2017 32nd IEEE\/ACM Int. conf. on automated software engineering (ASE), IEEE, pp 417\u2013427","DOI":"10.1109\/ASE.2017.8115654"},{"key":"10390_CR103","doi-asserted-by":"crossref","unstructured":"Scalabrino S, Linares-Vasquez M, Poshyvanyk D, Oliveto R (2016) Improving code readability models with textual features. In: 2016 IEEE 24th Int. Conf. on Program Comprehension (ICPC), IEEE, pp 1\u201310","DOI":"10.1109\/ICPC.2016.7503707"},{"key":"10390_CR104","unstructured":"Schults C (2021) Unit testing best practices: 9 to ensure you do it right. Blog, https:\/\/www.testim.io\/blog\/unit-testing-best-practices\/"},{"key":"10390_CR105","doi-asserted-by":"publisher","first-page":"169036","DOI":"10.1109\/ACCESS.2020.3022876","volume":"8","author":"N Setiani","year":"2020","unstructured":"Setiani N, Ferdiana R, Hartanto R (2020) Test case understandability model. IEEE. Access 8:169036\u2013169046","journal-title":"Access"},{"key":"10390_CR106","doi-asserted-by":"crossref","unstructured":"Setiani N, Ferdiana R, Hartanto R (2021a) Developer\u2019s perspectives on unit test cases understandability. In: Proceedings of the IEEE int. conf. on software engineering and service sciences, ICSESS, IEEE computer society, vol 2021-August, pp 251\u2013255","DOI":"10.1109\/ICSESS52187.2021.9522166"},{"key":"10390_CR107","doi-asserted-by":"crossref","unstructured":"Setiani N, Ferdiana R, Hartanto R (2021b) Developer\u2019s perspectives on unit test cases understandability. In: Proceedings of the IEEE Int. conf. on software engineering and service sciences, ICSESS, IEEE Computer Society, vol 2021-August, pp 251\u2013255","DOI":"10.1109\/ICSESS52187.2021.9522166"},{"key":"10390_CR108","doi-asserted-by":"publisher","unstructured":"Shamshiri S, Rojas JM, Galeotti JP, Walkinshaw N, Fraser G (2018) How do automatically generated unit tests influence software maintenance? In: 2018 IEEE 11th International conference on software testing, verification and validation (ICST), pp 250\u2013261, https:\/\/doi.org\/10.1109\/ICST.2018.00033","DOI":"10.1109\/ICST.2018.00033"},{"key":"10390_CR109","unstructured":"Shih J (2020) A field guide to unit testing: readability. Blog, https:\/\/codecharms.me\/posts\/unit-testing-readability"},{"key":"10390_CR110","unstructured":"Smart JF (2020) What makes a great test automation framework? - linkedin. Blog, https:\/\/www.linkedin.com\/pulse\/what-makes-great-test-automation-framework-john-ferguson-smart"},{"key":"10390_CR111","unstructured":"Snyder D, Kuefler E (2019) Testing on the toilet: tests too dry? make them damp! Blog, https:\/\/testing.googleblog.com\/2019\/12\/testing-on-toilet-tests-too-dry-make.html"},{"key":"10390_CR112","doi-asserted-by":"crossref","unstructured":"Spadini D, Palomba F, Zaidman A, Bruntink M, Bacchelli A (2018) On the relation of test smells to software code quality. In: 2018 IEEE international conference on software maintenance and evolution (ICSME), IEEE, pp 1\u201312","DOI":"10.1109\/ICSME.2018.00010"},{"key":"10390_CR113","unstructured":"testing company (2020) Unit testing tutorial: 5 best practices - vtest blog. Blog, https:\/\/www.vtestcorp.com\/blog\/unit-testing-best-practices\/"},{"key":"10390_CR114","doi-asserted-by":"crossref","unstructured":"Tran HKV, Ali NB, B\u00f6rstler J, Unterkalmsteiner M (2019) Test-case quality-understanding practitioners\u2019 perspectives. Int. Springer, Conf. on product-focused software process improvement, pp 37\u201352","DOI":"10.1007\/978-3-030-35333-9_3"},{"key":"10390_CR115","doi-asserted-by":"crossref","unstructured":"Tufano M, Palomba F, Bavota G, Di\u00a0Penta M, Oliveto R, De\u00a0Lucia A, Poshyvanyk D (2016) An empirical investigation into the nature of test smells. In: Proceedings of the 31st IEEE\/ACM international conference on automated software engineering, pp 4\u201315","DOI":"10.1145\/2970276.2970340"},{"key":"10390_CR116","doi-asserted-by":"crossref","unstructured":"Vahabzadeh A, Fard AM, Mesbah A (2015) An empirical study of bugs in test code. In: 2015 IEEE international conference on software maintenance and evolution (ICSME), IEEE, pp 101\u2013110","DOI":"10.1109\/ICSM.2015.7332456"},{"key":"10390_CR117","unstructured":"Van\u00a0Deursen A, Moonen L, Van Den\u00a0Bergh A, Kok G (2001) Refactoring test code. In: Proceedings of the 2nd international conference on extreme programming and flexible processes in software engineering (XP2001), Citeseer, pp 92\u201395"},{"key":"10390_CR118","unstructured":"Vance S (2013) Quality Code: software testing principles, practices, and ... nan, https:\/\/vdoc.pub\/documents\/quality-code-software-testing-principles-practices-and-patterns-781g7idb34f0"},{"key":"10390_CR119","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1016\/j.infsof.2015.02.009","volume":"62","author":"S Wagner","year":"2015","unstructured":"Wagner S, Goeb A, Heinemann L, Kl\u00e4s M, Lampasona C, Lochmann K, Mayr A, Pl\u00f6sch R, Seidl A, Streit J et al (2015) Operationalised product quality models and assessment: the quamoco approach. Information and Software Technology 62:101\u2013123","journal-title":"Information and Software Technology"},{"key":"10390_CR120","doi-asserted-by":"crossref","unstructured":"Wagner S, Lochmann K, Heinemann L, Kl\u00e4s M, Trendowicz A, Pl\u00f6sch R, Seidi A, Goeb A, Streit J (2012) The quamoco product quality modelling and assessment approach. In: 2012 34th international conference on software engineering (ICSE), IEEE, pp 1133\u20131142","DOI":"10.1109\/ICSE.2012.6227106"},{"key":"10390_CR121","doi-asserted-by":"crossref","unstructured":"Winkler D, Urbanke P, Ramler R (2021) What do we know about readability of test code? - a systematic mapping study. In: Proceedings of the 5th workshop on validation, analysis, and evolution of software tests, in conjunction with the 29th IEEE international conference on software analysis, evolution, and reengineering (SANER)","DOI":"10.1109\/SANER53432.2022.00135"},{"key":"10390_CR122","doi-asserted-by":"publisher","unstructured":"Winkler D, Urbanke P, Ramler R (2023) Data set for \"investigating the readability of test code: combining scientific and practical views\". Data Set, https:\/\/doi.org\/10.48436\/w4q8v-28695, https:\/\/doi.org\/10.48436\/w4q8v-28695","DOI":"10.48436\/w4q8v-28695"},{"key":"10390_CR123","doi-asserted-by":"crossref","unstructured":"Wohlin C, Runeson P, H\u00f6st M, Ohlsson MC, Regnell B, Wessl\u00e9n A (2012) Experimentation in software engineering. Springer Science & Business Media","DOI":"10.1007\/978-3-642-29044-2"},{"key":"10390_CR124","doi-asserted-by":"crossref","unstructured":"Yetistiren B, Ozsoy I, Tuzun E (2022) Assessing the quality of github copilot\u2019s code generation. In: Proceedings of the 18th international conference on predictive models and data analytics in software engineering, pp 62\u201371","DOI":"10.1145\/3558489.3559072"},{"key":"10390_CR125","unstructured":"Yonekubo T (2021) Readable test code - medium. Blog, https:\/\/medium.com\/@t-yonekubo\/readable-test-code-cad8a7babc7b"},{"key":"10390_CR126","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1016\/j.infsof.2014.06.009","volume":"58","author":"VG Yusifo\u011flu","year":"2015","unstructured":"Yusifo\u011flu VG, Amannejad Y, Can AB (2015) Software test-code engineering: a systematic mapping. Information and Software Technology 58:123\u2013147","journal-title":"Information and Software Technology"},{"issue":"3","key":"10390_CR127","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1007\/s10664-010-9143-7","volume":"16","author":"A Zaidman","year":"2011","unstructured":"Zaidman A, Van Rompaey B, van Deursen A, Demeyer S (2011) Studying the co-evolution of production and test code in open source and industrial developer test processes through repository mining. Empirical Software Engineering 16(3):325\u2013364","journal-title":"Empirical Software Engineering"},{"key":"10390_CR128","doi-asserted-by":"crossref","unstructured":"Zhang B, Hill E, Clause J (2016) Towards automatically generating descriptive names for unit tests. In: ASE 2016 - Proceedings of the 31st IEEE\/ACM int. conf. on automated software engineering, Association for Computing Machinery, Inc, pp 625\u2013636","DOI":"10.1145\/2970276.2970342"},{"key":"10390_CR129","unstructured":"Zilberfeld G (2014) Test attribute #2: Readability - java code geeks. Blog, https:\/\/www.javacodegeeks.com\/2014\/07\/test-attribute-2-readability.html"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10390-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-023-10390-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10390-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,23]],"date-time":"2024-03-23T02:25:58Z","timestamp":1711160758000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-023-10390-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,26]]},"references-count":129,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,3]]}},"alternative-id":["10390"],"URL":"https:\/\/doi.org\/10.1007\/s10664-023-10390-z","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,26]]},"assertion":[{"value":"30 August 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 February 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declaration"}},{"value":"None","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of interest"}}],"article-number":"53"}}