{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T15:03:25Z","timestamp":1770995005359,"version":"3.50.1"},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2020,9,28]],"date-time":"2020-09-28T00:00:00Z","timestamp":1601251200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003495","name":"Hessisches Ministerium f\u00fcr Wissenschaft und Kunst","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003495","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Priv. Secur."],"published-print":{"date-parts":[[2021,2,28]]},"abstract":"<jats:p>\n            In this article, we investigate a fundamental question regarding software security:\n            <jats:italic>Is the security of SW releases increasing over time?<\/jats:italic>\n            We approach this question with a detailed analysis of the large body of open-source software packaged in the popular Debian GNU\/Linux distribution. Contrary to common intuition, we find no clear evidence that the vulnerability rate of widely used software decreases over time: Even in popular and \u201cstable\u201d releases, the fixing of bugs does not seem to reduce the rate of newly identified vulnerabilities. The intuitive conclusion is worrisome: Commonly employed development and validation procedures do not seem to scale with the increase of features and complexity\u2014they are only chopping pieces off the top of an iceberg of vulnerabilities.\n          <\/jats:p>\n          <jats:p>\n            To the best of our knowledge, this is the first investigation into the problem that studies a complete distribution of software, spanning multiple versions. Although we can not give a definitive answer, we show that several popular beliefs also cannot be confirmed given our dataset. We publish our\n            <jats:italic>Debian Vulnerability Analysis Framework (DVAF)<\/jats:italic>\n            , an automated dataset creation and analysis process, to enable reproduction and further analysis of our results. Overall, we hope our contributions provide important insights into the vulnerability discovery process and help in identifying effective techniques for vulnerability analysis and prevention.\n          <\/jats:p>","DOI":"10.1145\/3406112","type":"journal-article","created":{"date-parts":[[2020,9,28]],"date-time":"2020-09-28T11:02:35Z","timestamp":1601290955000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["The Tip of the Iceberg"],"prefix":"10.1145","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8383-4761","authenticated-orcid":false,"given":"Nikolaos","family":"Alexopoulos","sequence":"first","affiliation":[{"name":"Technical University of Darmstadt, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sheikh Mahbub","family":"Habib","sequence":"additional","affiliation":[{"name":"Continental AG, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Steffen","family":"Schulz","sequence":"additional","affiliation":[{"name":"Intel Labs, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Max","family":"M\u00fchlh\u00e4user","sequence":"additional","affiliation":[{"name":"Technical University of Darmstadt, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,9,28]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Debian Project. 2016. Debian security FAQ. Retrieved from https:\/\/www.debian.org\/security\/faq.  Debian Project. 2016. Debian security FAQ. Retrieved from https:\/\/www.debian.org\/security\/faq."},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the Annual Reliability and Maintainability Symposium. IEEE, 615--620","author":"Alhazmi O. H."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133956.3133960"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0085777"},{"key":"e_1_2_1_5_1","volume-title":"Measuring Lenny: The size of Debian 5.0.","author":"Amor Juan Jos\u00e9","year":"2009"},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the 31st IEEE Symposium on Security and Privacy (S8P\u201910)","author":"Bau Jason","year":"2010"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2382196.2382284"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 26th USENIX Security Symposium (USENIX Security\u201917)","author":"Biswas Priyam","year":"2017"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866307.1866337"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.68"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2046582.2046587"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the 26th Annual Computer Security Applications Conference (ACSAC\u201910)","author":"Clark Sandy","year":"1920"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1137\/070710111"},{"key":"e_1_2_1_14_1","unstructured":"CVE Details. 2020. Browse Vulnerabilities by Date. Retrieved from https:\/\/www.cvedetails.com\/browse-by-date.php.  CVE Details. 2020. Browse Vulnerabilities by Date. Retrieved from https:\/\/www.cvedetails.com\/browse-by-date.php."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2382196.2382218"},{"key":"e_1_2_1_16_1","volume-title":"hospital breach biggest yet to exploit Heartbleed bug: Expert","author":"Finkle Jim","year":"2014"},{"key":"e_1_2_1_17_1","volume-title":"Hackers are already using the Shellshock bug to launch botnet attacks. Wired","author":"Greenberg Andy","year":"2014"},{"key":"e_1_2_1_18_1","volume-title":"The Hacker-powered security report","year":"2017"},{"key":"e_1_2_1_19_1","first-page":"1920","article-title":"Game of detections: How are security vulnerabilities discovered in the wild?Empir","volume":"21","author":"Hafiz Munawar","year":"2016","journal-title":"Softw. Eng."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/HASE.2007.78"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/PROC.1980.11805"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/METRIC.1997.637156"},{"key":"e_1_2_1_23_1","first-page":"265","article-title":"Software challenges in achieving space safety","volume":"62","author":"Leveson Nancy G.","year":"2009","journal-title":"J. Brit. Interplan. Soc."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133956.3134072"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1093\/cybsec\/tyx008"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2015.48"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3134600.3134612"},{"key":"e_1_2_1_28_1","volume-title":"Schechter","author":"Ozment Andy","year":"2006"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the 26th USENIX Security Symposium (USENIX Security\u201917)","author":"Pan Jianfeng","year":"2017"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2810103.2813604"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2005.17"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2015.03.003"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the 26th USENIX Security Symposium (USENIX Security\u201917)","author":"Schumilo Sergej","year":"2017"},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of the USENIX Annual Technical Conference, Gernot Heiser and Wilson C. Hsieh (Eds.). USENIX Association, 309--318","author":"Serebryany Konstantin","year":"2012"},{"key":"e_1_2_1_36_1","volume-title":"Proceedings of the 34th International Conference on Software Engineering (ICSE\u201912)","author":"Shahzad Muhammad","year":"2012"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2016.23368"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-013-9258-8"},{"key":"e_1_2_1_40_1","volume-title":"Proceedings of the 28th USENIX Security Symposium (USENIX Security\u201919)","author":"Vahldiek-Oberwagner Anjo","year":"2019"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2994539.2994542"},{"key":"e_1_2_1_42_1","volume-title":"Taqqu","author":"Willinger Walter","year":"1998"},{"key":"e_1_2_1_43_1","volume-title":"Proceedings of the 27th USENIX Security Symposium (USENIX Security\u201918)","author":"Xiao Chaowei","year":"2018"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2420950.2421003"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2810103.2813704"}],"container-title":["ACM Transactions on Privacy and Security"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3406112","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3406112","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:31:52Z","timestamp":1750195912000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3406112"}},"subtitle":["On the Merits of Finding Security Bugs"],"short-title":[],"issued":{"date-parts":[[2020,9,28]]},"references-count":43,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,2,28]]}},"alternative-id":["10.1145\/3406112"],"URL":"https:\/\/doi.org\/10.1145\/3406112","relation":{},"ISSN":["2471-2566","2471-2574"],"issn-type":[{"value":"2471-2566","type":"print"},{"value":"2471-2574","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,28]]},"assertion":[{"value":"2019-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-09-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}