{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,4]],"date-time":"2026-11-04T18:02:15Z","timestamp":1793815335578,"version":"4.6.1"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2023,4,17]],"date-time":"2023-04-17T00:00:00Z","timestamp":1681689600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,4,17]],"date-time":"2023-04-17T00:00:00Z","timestamp":1681689600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["402774445"],"award-info":[{"award-number":["402774445"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100016135","name":"Universit\u00e4t Passau","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100016135","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2023,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Automated Static Analysis Tools (ASATs) are part of software development best practices. ASATs are able to warn developers about potential problems in the code. On the one hand, ASATs are based on best practices so there should be a noticeable effect on software quality. On the other hand, ASATs suffer from false positive warnings, which developers have to inspect and then ignore or mark as invalid. In this article, we ask whether ASATs have a measurable impact on external software quality, using the example of PMD for Java. We investigate the relationship between ASAT warnings emitted by PMD on defects per change and per file. Our case study includes data for the history of each file as well as the differences between changed files and the project in which they are contained. We investigate whether files that induce a defect have more static analysis warnings than the rest of the project. Moreover, we investigate the impact of two different sets of ASAT rules. We find that, bug inducing files contain less static analysis warnings than other files of the project at that point in time. However, this can be explained by the overall decreasing warning density. When compared with all other changes, we find a statistically significant difference in one metric for all rules and two metrics for a subset of rules. However, the effect size is negligible in all cases, showing that the actual difference in warning density between bug inducing changes and other changes is small at best.<\/jats:p>","DOI":"10.1007\/s10664-023-10301-2","type":"journal-article","created":{"date-parts":[[2023,4,17]],"date-time":"2023-04-17T09:09:04Z","timestamp":1681722544000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Are automated static analysis tools worth it? An investigation into relative warning density and external software quality on the example of Apache open source projects"],"prefix":"10.1007","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5236-7953","authenticated-orcid":false,"given":"Alexander","family":"Trautsch","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9765-2803","authenticated-orcid":false,"given":"Steffen","family":"Herbold","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jens","family":"Grabowski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,4,17]]},"reference":[{"key":"10301_CR1","unstructured":"Abdi H (2007) Bonferroni and Sidak corrections for multiple comparisons. In: Encyclopedia of measurement and statistics. Sage, Thousand Oaks, pp 103\u2013107"},{"key":"10301_CR2","doi-asserted-by":"publisher","first-page":"110427","DOI":"10.1016\/j.jss.2019.110427","volume":"158","author":"B Aloraini","year":"2019","unstructured":"Aloraini B, Nagappan M, German DM, Hayashi S, Higo Y (2019) An empirical study of security warnings from static application security testing tools. J Syst Softw 158:110427. https:\/\/doi.org\/10.1016\/j.jss.2019.110427. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0164121219302018","journal-title":"J Syst Softw"},{"key":"10301_CR3","doi-asserted-by":"publisher","unstructured":"Antoniol G, Ayari K, Di Penta M, Khomh F, Gu\u00e9h\u00e9neuc YG (2008) Is it a bug or an enhancement? a text-based approach to classify change requests. In: Proceedings of the 2008 conference of the center for advanced studies on collaborative research: Meeting of minds, CASCON \u201908. Association for Computing Machinery, New York. https:\/\/doi.org\/10.1145\/1463788.1463819","DOI":"10.1145\/1463788.1463819"},{"key":"10301_CR4","doi-asserted-by":"publisher","unstructured":"Christakis M, Bird C (2016) What developers want and need from program analysis: An empirical study. In: Proceedings of the 31st IEEE\/ACM international conference on automated software engineering, ASE 2016. ACM, New York, pp 332\u2013343. https:\/\/doi.org\/10.1145\/2970276.2970347","DOI":"10.1145\/2970276.2970347"},{"key":"10301_CR5","doi-asserted-by":"publisher","first-page":"494","DOI":"10.1037\/0033-2909.114.3.494","volume":"114","author":"N Cliff","year":"1993","unstructured":"Cliff N (1993) Dominance statistics: Ordinal analyses to answer ordinal questions. Psychol Bull 114:494\u2013509","journal-title":"Psychol Bull"},{"key":"10301_CR6","doi-asserted-by":"publisher","unstructured":"Devanbu P, Zimmermann T, Bird C (2016) Belief evidence in empirical software engineering. In: 2016 IEEE\/ACM 38th international conference on software engineering (ICSE), pp 108\u2013119. https:\/\/doi.org\/10.1145\/2884781.2884812","DOI":"10.1145\/2884781.2884812"},{"key":"10301_CR7","doi-asserted-by":"publisher","unstructured":"Habib A, Pradel M (2018) How many of all bugs do we find? a study of static bug detectors. In: Proceedings of the 33rd ACM\/IEEE international conference on automated software engineering, ASE 2018. ACM, New York, pp 317\u2013328. https:\/\/doi.org\/10.1145\/3238147.3238213","DOI":"10.1145\/3238147.3238213"},{"key":"10301_CR8","doi-asserted-by":"publisher","unstructured":"Heckman S, Williams L (2009) A model building process for identifying actionable static analysis alerts. In: 2009 international conference on software testing verification and validation, pp 161\u2013170. https:\/\/doi.org\/10.1109\/ICST.2009.45","DOI":"10.1109\/ICST.2009.45"},{"key":"10301_CR9","doi-asserted-by":"publisher","unstructured":"Herbold S, Trautsch A, Ledel B, Aghamohammadi A, Ghaleb TA, Chahal KK, Bossenmaier T, Nagaria B, Makedonski P, Ahmadabadi MN, Szabados K, Spieker H, Madeja M, Hoy N, Lenarduzzi V, Wang S, Rodr\u00edguez-P\u00e9rez G, Colomo-Palacios R, Verdecchia R, Singh P, Qin Y, Chakroborti D, Davis W, Walunj V, Wu H, Marcilio D, Alam O, Aldaeej A, Amit I, Turhan B, Eismann S, Wickert AK, Malavolta I, Sulir M, Fard F, Henley AZ, Kourtzanidis S, Tuzun E, Treude C, Shamasbi SM, Pashchenko I, Wyrich M, Davis J, Serebrenik A, Albrecht E, Aktas EU, Str\u00fcber D, Erbel J (2022a) A fine-grained data set and analysis of tangling in bug fixing commits. Empir Softw Eng 27(6):125. https:\/\/doi.org\/10.1007\/s10664-021-10083-5","DOI":"10.1007\/s10664-021-10083-5"},{"key":"10301_CR10","doi-asserted-by":"publisher","unstructured":"Herbold S, Trautsch A, Trautsch F, Ledel B (2022b) Problems with SZZ and features: An empirical study of the state of practice of defect prediction data collection. Empir Softw Eng 27(2):42. https:\/\/doi.org\/10.1007\/s10664-021-10092-4","DOI":"10.1007\/s10664-021-10092-4"},{"key":"10301_CR11","doi-asserted-by":"crossref","unstructured":"Herzig K, Just S, Zeller A (2013) It\u2019s not a bug, it\u2019s a feature: How misclassification impacts bug prediction. In: Proceedings of the 2013 international conference on software engineering, ICSE \u201913. IEEE Press, pp 392\u2013401","DOI":"10.1109\/ICSE.2013.6606585"},{"key":"10301_CR12","doi-asserted-by":"crossref","unstructured":"Johnson B, Song Y, Murphy-Hill E, Bowdidge R (2013) Why don't software developers use static analysis tools to find bugs?. In: Proceedings of the 2013 international conference on software engineering, ICSE \u201913. IEEE Press, Piscataway, pp 672\u2013681. http:\/\/dl.acm.org\/citation.cfm?id=2486788.2486877","DOI":"10.1109\/ICSE.2013.6606613"},{"key":"10301_CR13","doi-asserted-by":"publisher","unstructured":"Just R, Jalali D, Ernst MD (2014) Defects4J: A database of existing faults to enable controlled testing studies for java programs. In: Proceedings of the 2014 international symposium on software testing and analysis, ISSTA 2014. Association for Computing Machinery, New York, pp 437\u2013440. https:\/\/doi.org\/10.1145\/2610384.2628055","DOI":"10.1145\/2610384.2628055"},{"issue":"6","key":"10301_CR14","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1109\/TSE.2012.70","volume":"39","author":"Y Kamei","year":"2013","unstructured":"Kamei Y, Shihab E, Adams B, Hassan AE, Mockus A, Sinha A, Ubayashi N (2013) A large-scale empirical study of just-in-time quality assurance. IEEE Trans Softw Eng 39(6):757\u2013773. https:\/\/doi.org\/10.1109\/TSE.2012.70","journal-title":"IEEE Trans Softw Eng"},{"key":"10301_CR15","doi-asserted-by":"publisher","unstructured":"Kim S, Ernst MD (2007) Which warnings should I fix first?. In: Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering, ESEC-FSE \u201907. ACM, New York, pp 45\u201354. https:\/\/doi.org\/10.1145\/1287624.1287633","DOI":"10.1145\/1287624.1287633"},{"key":"10301_CR16","doi-asserted-by":"publisher","unstructured":"Koc U, Saadatpanah P, Foster JS, Porter AA (2017) Learning a classifier for false positive error reports emitted by static code analysis tools. In: Proceedings of the 1st ACM SIGPLAN international workshop on machine learning and programming languages, pp 35\u201342. https:\/\/doi.org\/10.1145\/3088525.3088675","DOI":"10.1145\/3088525.3088675"},{"key":"10301_CR17","doi-asserted-by":"crossref","unstructured":"Lenarduzzi V, Lomio F, Huttunen H, Taibi D (2020) Are sonarqube rules inducing bugs? 2020 IEEE 27th international conference on software analysis, evolution and reengineering (SANER), pp 501\u2013511","DOI":"10.1109\/SANER48275.2020.9054821"},{"issue":"1","key":"10301_CR18","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1214\/aoms\/1177730491","volume":"18","author":"HB Mann","year":"1947","unstructured":"Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 18(1):50\u201360","journal-title":"Ann Math Stat"},{"key":"10301_CR19","doi-asserted-by":"publisher","unstructured":"Nagappan N, Ball T (2005) Static analysis tools as early indicators of pre-release defect density. In: Proceedings of the 27th international conference on software engineering, ICSE \u201905. ACM, New York, pp 580\u2013586. https:\/\/doi.org\/10.1145\/1062455.1062558","DOI":"10.1145\/1062455.1062558"},{"key":"10301_CR20","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1016\/j.jss.2018.12.001","volume":"150","author":"L Pascarella","year":"2019","unstructured":"Pascarella L, Palomba F, Bacchelli A (2019) Fine-grained just-in-time defect prediction. J Syst Softw 150:22\u201336. https:\/\/doi.org\/10.1016\/j.jss.2018.12.001, http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0164121218302656","journal-title":"J Syst Softw"},{"issue":"10","key":"10301_CR21","doi-asserted-by":"publisher","first-page":"1469","DOI":"10.1016\/j.infsof.2009.04.013","volume":"51","author":"MD Penta","year":"2009","unstructured":"Penta MD, Cerulo L, Aversano L (2009) The life and death of statically detected vulnerabilities: An empirical study. Inf Softw Technol 51(10):1469\u20131484. https:\/\/doi.org\/10.1016\/j.infsof.2009.04.013. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0950584909000500, source Code Analysis and Manipulation, SCAM 2008","journal-title":"Inf Softw Technol"},{"key":"10301_CR22","doi-asserted-by":"publisher","unstructured":"Plosch R, Gruber H, Hentschel A, Pomberger G, Schiffer S (2008) On the relation between external software quality and static code analysis. In: 2008 32nd annual IEEE software engineering workshop, pp 169\u2013174. https:\/\/doi.org\/10.1109\/SEW.2008.17","DOI":"10.1109\/SEW.2008.17"},{"key":"10301_CR23","doi-asserted-by":"publisher","unstructured":"Querel L, Rigby PC (2021) Warning-introducing commits vs bug-introducing commits: A tool, statistical models, and a preliminary user study. In: 29th IEEE\/ACM International Conference on Program Comprehension, ICPC 2021, Madrid, Spain, May 20-21, 2021. IEEE, pp 433\u2013443. https:\/\/doi.org\/10.1109\/ICPC52881.2021.00051","DOI":"10.1109\/ICPC52881.2021.00051"},{"key":"10301_CR24","doi-asserted-by":"publisher","unstructured":"Querel LP, Rigby PC (2018) Warningsguru: Integrating statistical bug models with static analysis to provide timely and specific bug warnings. In: Proceedings of the 2018 26th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, ESEC\/FSE 2018. Association for Computing Machinery, New York, pp 892\u2013895. https:\/\/doi.org\/10.1145\/3236024.3264599","DOI":"10.1145\/3236024.3264599"},{"key":"10301_CR25","doi-asserted-by":"publisher","unstructured":"Rahman F, Khatri S, Barr ET, Devanbu P (2014) Comparing static bug finders and statistical prediction. In: Proceedings of the 36th international conference on software engineering, ICSE 2014. ACM, New York, pp 424\u2013434. https:\/\/doi.org\/10.1145\/2568225.2568269","DOI":"10.1145\/2568225.2568269"},{"key":"10301_CR26","doi-asserted-by":"publisher","unstructured":"Rodriguez-P\u00e9rez G, Nagappan M, Robles G (2020) Watch out for extrinsic bugs! a case study of their impact in just-in-time bug prediction models on the openstack project. IEEE Trans Softw Eng 1\u20131. https:\/\/doi.org\/10.1109\/TSE.2020.3021380","DOI":"10.1109\/TSE.2020.3021380"},{"key":"10301_CR27","doi-asserted-by":"publisher","unstructured":"Rosen C, Grawi B, Shihab E (2015) Commit guru: Analytics and risk prediction of software commits. In: Proceedings of the 2015 10th joint meeting on foundations of software engineering, ESEC\/FSE 2015. Association for Computing Machinery, New York, pp 966\u2013969. https:\/\/doi.org\/10.1145\/2786805.2803183","DOI":"10.1145\/2786805.2803183"},{"key":"10301_CR28","doi-asserted-by":"publisher","unstructured":"Spadini D, Aniche M, Bacchelli A (2018) PyDriller: Python framework for mining software repositories. In: Proceedings of the 2018 26th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering - ESEC\/FSE 2018. ACM Press, New York, pp 908\u2013911. https:\/\/doi.org\/10.1145\/3236024.3264598, http:\/\/dl.acm.org\/citation.cfm?doid=3236024.3264598","DOI":"10.1145\/3236024.3264598"},{"key":"10301_CR29","doi-asserted-by":"publisher","unstructured":"Thung F, Lucia Lo D, Jiang L, Rahman F, Devanbu PT (2012) To what extent could we detect field defects? an empirical study of false negatives in static bug finding tools. In: 2012 Proceedings of the 27th IEEE\/ACM international conference on automated software engineering, pp 50\u201359. https:\/\/doi.org\/10.1145\/2351676.2351685","DOI":"10.1145\/2351676.2351685"},{"key":"10301_CR30","doi-asserted-by":"publisher","unstructured":"Trautsch A, Herbold S, Grabowski J (2020a) A longitudinal study of static analysis warning evolution and the effects of PMD on software quality in apache open source projects. Empirical Software Engineering. https:\/\/doi.org\/10.1007\/s10664-020-09880-1","DOI":"10.1007\/s10664-020-09880-1"},{"key":"10301_CR31","doi-asserted-by":"crossref","unstructured":"Trautsch A, Herbold S, Grabowski J (2020) Static source code metrics and static analysis warnings for fine-grained just-in-time defect prediction. In: 36th international conference on software maintenance and evolution (ICSME)","DOI":"10.1109\/ICSME46990.2020.00022"},{"key":"10301_CR32","doi-asserted-by":"publisher","unstructured":"Trautsch F, Herbold S, Makedonski P, Grabowski J (2017) Addressing problems with replicability and validity of repository mining studies through a smart data platform. Empirical Software Engineering. https:\/\/doi.org\/10.1007\/s10664-017-9537-x","DOI":"10.1007\/s10664-017-9537-x"},{"key":"10301_CR33","doi-asserted-by":"crossref","unstructured":"Tufano M, Palomba F, Bavota G, Penta MD, Oliveto R, Lucia AD, Poshyvanyk D (2017) There and back again: Can you compile that snapshot? J Softw Evol Process 29(4). http:\/\/dblp.uni-trier.de\/db\/journals\/smr\/smr29.html#TufanoPBPOLP17","DOI":"10.1002\/smr.1838"},{"key":"10301_CR34","doi-asserted-by":"publisher","unstructured":"Vassallo C, Panichella S, Palomba F, Proksch S, Gall HC, Zaidman A (2020) How developers engage with static analysis tools in different contexts. Empir Softw Eng 25. https:\/\/doi.org\/10.1007\/s10664-019-09750-5","DOI":"10.1007\/s10664-019-09750-5"},{"key":"10301_CR35","doi-asserted-by":"publisher","unstructured":"Vetro A, Morisio M, Torchiano M (2011) An empirical validation of findbugs issues related to defects. In: 15th annual conference on evaluation assessment in software engineering (EASE 2011), pp 144\u2013153. https:\/\/doi.org\/10.1049\/ic.2011.0018","DOI":"10.1049\/ic.2011.0018"},{"key":"10301_CR36","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4615-4625-2","volume-title":"Experimentation in software engineering: an introduction","author":"C Wohlin","year":"2000","unstructured":"Wohlin C, Runeson P, H\u00f6st M, Ohlsson MC, Regnell B, Wessl\u00e9n A (2000) Experimentation in software engineering: an introduction. Kluwer Academic Publishers, Norwell"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10301-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-023-10301-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10301-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,26]],"date-time":"2023-05-26T21:03:01Z","timestamp":1685134981000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-023-10301-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,17]]},"references-count":36,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,5]]}},"alternative-id":["10301"],"URL":"https:\/\/doi.org\/10.1007\/s10664-023-10301-2","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,17]]},"assertion":[{"value":"3 February 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 April 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no competing interests to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Competing interests"}}],"article-number":"66"}}