{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T06:28:00Z","timestamp":1770272880762,"version":"3.49.0"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2022,3,17]],"date-time":"2022-03-17T00:00:00Z","timestamp":1647475200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,17]],"date-time":"2022-03-17T00:00:00Z","timestamp":1647475200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2022,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Test-to-code traceability links model the relationships between test artefacts and code artefacts. When utilised during the development process, these links help developers to keep test code in sync with tested code, reducing the rate of test failures and missed faults. Test-to-code traceability links can also help developers to maintain an accurate mental model of the system, reducing the risk of architectural degradation when making changes. However, establishing and maintaining these links manually places an extra burden on developers and is error-prone. This paper presents <jats:sc>TCTracer<\/jats:sc>, an approach and implementation for the automatic establishment of test-to-code traceability links. Unlike existing work, <jats:sc>TCTracer<\/jats:sc> operates at both the method level and the class level, allowing us to establish links between tests and functions, as well as between test classes and tested classes. We improve over existing techniques by combining an ensemble of new and existing techniques that utilise both dynamic and static information and exploiting a synergistic flow of information between the method and class levels. An evaluation of <jats:sc>TCTracer<\/jats:sc> using five large, well-studied open source systems demonstrates that, on average, we can establish test-to-function links with a mean average precision (MAP) of 85% and test-class-to-class links with an MAP of 92%.<\/jats:p>","DOI":"10.1007\/s10664-021-10079-1","type":"journal-article","created":{"date-parts":[[2022,3,17]],"date-time":"2022-03-17T14:06:53Z","timestamp":1647526013000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["TCTracer: Establishing test-to-code traceability links using dynamic and static techniques"],"prefix":"10.1007","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9579-7072","authenticated-orcid":false,"given":"Robert","family":"White","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1009-2861","authenticated-orcid":false,"given":"Jens","family":"Krinke","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,3,17]]},"reference":[{"key":"10079_CR1","doi-asserted-by":"publisher","first-page":"40111","DOI":"10.1109\/ACCESS.2021.3063158","volume":"9","author":"N Aljawabrah","year":"2021","unstructured":"Aljawabrah N, Gergely T, Misra S, Fernandez-Sanz L (2021) Automated recovery and visualization of test-to-code traceability (TCT) links: an evaluation. IEEE Access 9:40111\u201340123","journal-title":"IEEE Access"},{"issue":"4","key":"10079_CR2","first-page":"81","volume":"51","author":"M Allamanis","year":"2018","unstructured":"Allamanis M, Barr E T, Devanbu P, Sutton C (2018) A survey of machine learning for big code and naturalness. ACM Comput Surv (CSUR) 51 (4):81","journal-title":"ACM Comput Surv (CSUR)"},{"issue":"10","key":"10079_CR3","doi-asserted-by":"publisher","first-page":"970","DOI":"10.1109\/TSE.2002.1041053","volume":"28","author":"G Antoniol","year":"2002","unstructured":"Antoniol G, Canfora G, Casazza G, De Lucia A, Merlo E (2002) Recovering traceability links between code and documentation. IEEE Trans Softw Eng 28(10):970\u2013983","journal-title":"IEEE Trans Softw Eng"},{"key":"10079_CR4","doi-asserted-by":"publisher","unstructured":"Bouillon P, Krinke J, Meyer N, Steimann F (2007) EZUNIT: A framework for associating failed unit tests with potential programming errors. In: Agile Processes in Software Engineering and Extreme Programming. Springer. https:\/\/doi.org\/10.1007\/978-3-540-73101-6_14","DOI":"10.1007\/978-3-540-73101-6_14"},{"key":"10079_CR5","doi-asserted-by":"crossref","unstructured":"Cleland-Huang J (2012) Traceability in agile projects. In: Software and Systems Traceability. Springer, pp 265\u2013275","DOI":"10.1007\/978-1-4471-2239-5_12"},{"key":"10079_CR6","doi-asserted-by":"crossref","unstructured":"Csuvik V, Kicsi A, Vid\u00e1cs L (2019a) Evaluation of textual similarity techniques in code level traceability. In: International Conference on Computational Science and Its Applications. Springer, pp 529\u2013543","DOI":"10.1007\/978-3-030-24305-0_40"},{"key":"10079_CR7","doi-asserted-by":"publisher","unstructured":"Csuvik V, Kicsi A, Vid\u00e1cs L (2019b) Source code level word embeddings in aiding semantic test-to-code traceability. In: 10th International Workshop on Software and Systems Traceability, pp 29\u201336. https:\/\/doi.org\/10.1109\/SST.2019.00016","DOI":"10.1109\/SST.2019.00016"},{"key":"10079_CR8","doi-asserted-by":"crossref","unstructured":"Davis J, Goadrich M (2006) The relationship between precision-recall and ROC curves. In: Proceedings of the 23rd International Conference on Machine Learning. ACM, pp 233\u2013240","DOI":"10.1145\/1143844.1143874"},{"key":"10079_CR9","doi-asserted-by":"crossref","unstructured":"De Lucia A, Fasano F, Oliveto R (2008) Traceability management for impact analysis. In: 2008 Frontiers of Software Maintenance. IEEE, pp 21\u201330","DOI":"10.1109\/FOSM.2008.4659245"},{"key":"10079_CR10","doi-asserted-by":"crossref","unstructured":"Elsner D, Hauer F, Pretschner A, Reimer S (2021) Empirically evaluating readily available information for regression test optimization in continuous integration. In: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp 491\u2013504","DOI":"10.1145\/3460319.3464834"},{"issue":"2","key":"10079_CR11","doi-asserted-by":"publisher","first-page":"797","DOI":"10.1007\/s11219-018-9430-x","volume":"27","author":"T Gergely","year":"2019","unstructured":"Gergely T, Balogh G, Horv\u00e1th F, Vancsics B, Besz\u00e9des \u00c1, Gyim\u00f3thy T (2019) Differences between a static and a dynamic test-to-code traceability recovery method. Softw Qual J 27(2):797\u2013822","journal-title":"Softw Qual J"},{"key":"10079_CR12","doi-asserted-by":"crossref","unstructured":"Gethers M, Oliveto R, Poshyvanyk D, De Lucia A (2011) On integrating orthogonal information retrieval methods to improve traceability recovery. In: 27th IEEE International Conference on Software Maintenance (ICSM), pp 133\u2013142","DOI":"10.1109\/ICSM.2011.6080780"},{"key":"10079_CR13","doi-asserted-by":"publisher","unstructured":"Ghafari M, Ghezzi C, Rubinov K (2015) Automatically identifying focal methods under test in unit test cases. In: 15Th international working conference on source code analysis and manipulation (SCAM). IEEE, pp 61\u201370. https:\/\/doi.org\/10.1109\/SCAM.2015.7335402","DOI":"10.1109\/SCAM.2015.7335402"},{"key":"10079_CR14","doi-asserted-by":"publisher","unstructured":"Gold NE, Krinke J (2020) Ethical mining: A case study on MSR mining challenges. In: Proceedings of the 17th International Conference on Mining Software Repositories. ACM, pp 265\u2013276. https:\/\/doi.org\/10.1145\/3379597.3387462","DOI":"10.1145\/3379597.3387462"},{"key":"10079_CR15","doi-asserted-by":"publisher","unstructured":"Gold NE, Krinke J (2022) Ethics in the mining of software repositories. Empirical Software Engineering. 27(17) https:\/\/doi.org\/10.1007\/s10664-021-10057-7","DOI":"10.1007\/s10664-021-10057-7"},{"key":"10079_CR16","doi-asserted-by":"crossref","unstructured":"Hurdugaci V, Zaidman A (2012) Aiding software developers to maintain developer tests. In: 16th European Conference on Software Maintenance and Reengineering. IEEE, pp 11\u201320","DOI":"10.1109\/CSMR.2012.12"},{"key":"10079_CR17","doi-asserted-by":"crossref","unstructured":"Jones J A, Harrold M J, Stasko J (2002) Visualization of test information to assist fault localization. In: Proceedings of the 24th International Conference on Software Engineering (ICSE). IEEE, pp 467\u2013477","DOI":"10.1145\/581396.581397"},{"key":"10079_CR18","doi-asserted-by":"publisher","unstructured":"Kicsi A, T\u00f3th L, Vid\u00e1cs L (2018) Exploring the benefits of utilizing conceptual information in test-to-code traceability. In: 6th International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering, pp 8\u201314. https:\/\/doi.org\/10.1145\/3194104.3194106","DOI":"10.1145\/3194104.3194106"},{"key":"10079_CR19","doi-asserted-by":"publisher","unstructured":"Kicsi A, Vid\u00e1cs L, Gyim\u00f3thy T (2020) TestRoutes. In: Proceedings of the 17th International Conference on Mining Software Repositories. ACM, pp 593\u2013597. https:\/\/doi.org\/10.1145\/3379597.3387488","DOI":"10.1145\/3379597.3387488"},{"issue":"8","key":"10079_CR20","first-page":"707","volume":"10","author":"VI Levenshtein","year":"1966","unstructured":"Levenshtein V I (1966) Binary codes capable of correcting deletions, insertions, and reversals. Sov Phys Doklady 10(8):707\u2013710","journal-title":"Sov Phys Doklady"},{"key":"10079_CR21","doi-asserted-by":"publisher","unstructured":"Madeja M, Poruba\u0307n J (2019) Tracing naming semantics in unit tests of popular GitHub Android projects. In: 8th Symposium on Languages, Applications and Technologies, (SLATE 2019). Schloss Dagstuhl\u2013Leibniz-Zentrum fuer Informatik. https:\/\/doi.org\/10.4230\/OASIcs.SLATE.2019.3","DOI":"10.4230\/OASIcs.SLATE.2019.3"},{"issue":"1","key":"10079_CR22","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1017\/S1351324909005129","volume":"16","author":"C Manning","year":"2010","unstructured":"Manning C, Raghavan P, Sch\u00fctze H (2010) Introduction to information retrieval. Nat Lang Eng 16(1):100\u2013103","journal-title":"Nat Lang Eng"},{"key":"10079_CR23","doi-asserted-by":"publisher","unstructured":"Meimandi Parizi R, Kasem A, Abdullah A (2015) Towards gamification in software traceability: Between test and code artifacts. In: Proceedings of the 10th International Conference on Software Engineering and Applications. SCITEPRESS, pp 393\u2013400, https:\/\/doi.org\/10.5220\/0005555503930400","DOI":"10.5220\/0005555503930400"},{"key":"10079_CR24","doi-asserted-by":"publisher","unstructured":"Orellana G, Laghari G, Murgia A, Demeyer S (2017) On the differences between unit and integration testing in the TravisTorrent dataset. In: 2017 IEEE\/ACM 14th International Conference on Mining Software Repositories (MSR), pp 451\u2013454. https:\/\/doi.org\/10.1109\/MSR.2017.25","DOI":"10.1109\/MSR.2017.25"},{"key":"10079_CR25","doi-asserted-by":"publisher","unstructured":"Parizi R M (2016) On the gamification of human-centric traceability tasks in software testing and coding. In: IEEE 14th international conference on software engineering research, management and applications (SERA). IEEE, pp 193\u2013200. https:\/\/doi.org\/10.1109\/SERA.2016.7516146","DOI":"10.1109\/SERA.2016.7516146"},{"issue":"4","key":"10079_CR26","doi-asserted-by":"publisher","first-page":"913","DOI":"10.1109\/TR.2014.2338254","volume":"63","author":"RM Parizi","year":"2014","unstructured":"Parizi R M, Lee S P, Dabbagh M (2014) Achievements and challenges in state-of-the-art software traceability between test and code artifacts. IEEE Trans Reliab 63(4):913\u2013926. https:\/\/doi.org\/10.1109\/TR.2014.2338254","journal-title":"IEEE Trans Reliab"},{"key":"10079_CR27","doi-asserted-by":"publisher","unstructured":"Pinto L S, Sinha S, Orso A (2012) Understanding myths and realities of test-suite evolution. In: Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering (FSE). https:\/\/doi.org\/10.1145\/2393596.2393634, vol 1. ACM, p 1","DOI":"10.1145\/2393596.2393634"},{"issue":"11","key":"10079_CR28","doi-asserted-by":"publisher","first-page":"1167","DOI":"10.1002\/smr.1573","volume":"25","author":"A Qusef","year":"2013","unstructured":"Qusef A, Bavota G, Oliveto R, Lucia A D, Binkley D (2013) Evaluating test-to-code traceability recovery methods through controlled experiments. J Softw Evol Process 25(11):1167\u20131191. https:\/\/doi.org\/10.1002\/smr.1573","journal-title":"J Softw Evol Process"},{"key":"10079_CR29","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1016\/j.jss.2013.10.019","volume":"88","author":"A Qusef","year":"2014","unstructured":"Qusef A, Bavota G, Oliveto R, Lucia A D, Binkley D (2014) Recovering test-to-code traceability using slicing and textual analysis. J Syst Softw 88:147\u2013168","journal-title":"J Syst Softw"},{"key":"10079_CR30","doi-asserted-by":"publisher","first-page":"3909","DOI":"10.1109\/ACCESS.2017.2685629","volume":"5","author":"M Shahin","year":"2017","unstructured":"Shahin M, Babar M A, Zhu L (2017) Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices. IEEE Access 5:3909\u20133943","journal-title":"IEEE Access"},{"issue":"5","key":"10079_CR31","doi-asserted-by":"publisher","first-page":"1990","DOI":"10.1007\/s10664-015-9405-5","volume":"21","author":"QD Soetens","year":"2016","unstructured":"Soetens Q D, Demeyer S, Zaidman A, P\u0117rez J (2016) Change-based test selection: an empirical evaluation. Empir Softw Eng 21(5):1990\u20132032. https:\/\/doi.org\/10.1007\/s10664-015-9405-5","journal-title":"Empir Softw Eng"},{"issue":"3","key":"10079_CR32","doi-asserted-by":"publisher","first-page":"967","DOI":"10.1007\/s10664-016-9457-1","volume":"22","author":"D St\u00e5hl","year":"2017","unstructured":"St\u00e5hl D, Hall\u00e9n K, Bosch J (2017) Achieving traceability in large scale continuous integration and delivery deployment, usage and validation of the Eiffel framework. Empir Softw Eng 22(3):967\u2013995","journal-title":"Empir Softw Eng"},{"key":"10079_CR33","doi-asserted-by":"publisher","first-page":"110421","DOI":"10.1016\/j.jss.2019.110421","volume":"159","author":"F Trautsch","year":"2020","unstructured":"Trautsch F, Herbold S, Grabowski J (2020) Are unit and integration test definitions still valid for modern Java projects? an empirical study on open-source projects. J Syst Softw 159:110421","journal-title":"J Syst Softw"},{"key":"10079_CR34","unstructured":"UCL Research Ethics Committee (2020) Research Ethics at UCL. https:\/\/ethics.grad.ucl.ac.uk\/"},{"key":"10079_CR35","doi-asserted-by":"crossref","unstructured":"Van Rompaey B, Demeyer S (2009) Establishing traceability links between unit test cases and units under test. In: 13th European Conference on Software Maintenance and Reengineering. IEEE, pp 209\u2013218","DOI":"10.1109\/CSMR.2009.39"},{"key":"10079_CR36","doi-asserted-by":"crossref","unstructured":"Villmow J, Depoix J, Ulges A (2021) CONTEST: A unit test completion benchmark featuring context. In: The First Workshop on Natural Language Processing for Programming, pp 17\u201325","DOI":"10.18653\/v1\/2021.nlp4prog-1.2"},{"key":"10079_CR37","doi-asserted-by":"publisher","unstructured":"Watson C, Tufano M, Moran K, Bavota G, Poshyvanyk D (2020) On learning meaningful assert statements for unit test cases. In: 42Nd international conference on software engineering (ICSE), Seoul. https:\/\/doi.org\/10.1145\/3377811.3380429","DOI":"10.1145\/3377811.3380429"},{"key":"10079_CR38","doi-asserted-by":"publisher","unstructured":"White R, Krinke J (2018) TestNMT: Function-to-test neural machine translation. In: Proceedings of the 4th ACM SIGSOFT International Workshop on NLP for Software Engineering. https:\/\/doi.org\/10.1145\/3283812.3283823. ACM, NL4SE, pp 30\u201333","DOI":"10.1145\/3283812.3283823"},{"key":"10079_CR39","unstructured":"White R, Krinke J (2020) ReAssert: Deep learning for assert generation. arXiv:201109784"},{"key":"10079_CR40","doi-asserted-by":"publisher","unstructured":"White R, Krinke J (2021) TCTracer: Establishing test-to-code traceability links using dynamic and static techniques \u2013 evaluation data. https:\/\/doi.org\/10.5281\/zenodo.5206476","DOI":"10.5281\/zenodo.5206476"},{"key":"10079_CR41","doi-asserted-by":"publisher","unstructured":"White R, Krinke J, Tan R (2020) Establishing multilevel test-to-code traceability links In: 42nd International Conference on Software Engineering (ICSE). ACM. https:\/\/doi.org\/10.1145\/3377811.3380921","DOI":"10.1145\/3377811.3380921"},{"issue":"4","key":"10079_CR42","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1007\/s10270-009-0145-0","volume":"9","author":"S Winkler","year":"2010","unstructured":"Winkler S, von Pilgrim J (2010) A survey of traceability in requirements engineering and model-driven development. Softw Syst Model 9(4):529\u2013565","journal-title":"Softw Syst Model"},{"issue":"3","key":"10079_CR43","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1007\/s10664-010-9143-7","volume":"16","author":"A Zaidman","year":"2011","unstructured":"Zaidman A, Van Rompaey B, van Deursen A, Demeyer S (2011) Studying the co-evolution of production and test code in open source and industrial developer test processes through repository mining. Empir Softw Eng 16(3):325\u2013364. https:\/\/doi.org\/10.1007\/s10664-010-9143-7","journal-title":"Empir Softw Eng"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-10079-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-021-10079-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-10079-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,4,23]],"date-time":"2022-04-23T21:04:44Z","timestamp":1650747884000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-021-10079-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,17]]},"references-count":43,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,5]]}},"alternative-id":["10079"],"URL":"https:\/\/doi.org\/10.1007\/s10664-021-10079-1","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,17]]},"assertion":[{"value":"8 November 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 March 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"67"}}