{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,3]],"date-time":"2026-08-03T23:40:27Z","timestamp":1785800427887,"version":"3.56.0"},"reference-count":34,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2014,8,1]],"date-time":"2014-08-01T00:00:00Z","timestamp":1406851200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100003407","name":"Ministero dell'Istruzione, dell'Universit\u00e0 e della Ricerca","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003407","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004963","name":"Seventh Framework Programme","doi-asserted-by":"publisher","award":["285223"],"award-info":[{"award-number":["285223"]}],"id":[{"id":"10.13039\/501100004963","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst. Secur."],"published-print":{"date-parts":[[2014,8]]},"abstract":"<jats:p>\n            (U.S.) Rule-based policies for mitigating software risk suggest using the CVSS score to measure the risk of an individual vulnerability and act accordingly. A key issue is whether the \u2018danger\u2019 score does actually match the risk of exploitation in the wild, and if and how such a score could be improved. To address this question, we propose using a case-control study methodology similar to the procedure used to link lung cancer and smoking in the 1950s. A case-control study allows the researcher to draw conclusions on the relation between some\n            <jats:italic>risk factor<\/jats:italic>\n            (e.g., smoking) and an effect (e.g., cancer) by looking backward at the\n            <jats:italic>cases<\/jats:italic>\n            (e.g., patients) and comparing them with\n            <jats:italic>controls<\/jats:italic>\n            (e.g., randomly selected patients with similar characteristics). The methodology allows us to quantify the\n            <jats:italic>risk reduction<\/jats:italic>\n            achievable by acting on the risk factor. We illustrate the methodology by using publicly available data on vulnerabilities, exploits, and exploits in the wild to (1) evaluate the performances of the current risk factor in the industry, the CVSS base score; (2) determine whether it can be improved by considering additional factors such the existence of a proof-of-concept exploit, or of an exploit in the black markets. Our analysis reveals that (a) fixing a vulnerability just because it was assigned a high CVSS score is equivalent to randomly picking vulnerabilities to fix; (b) the existence of proof-of-concept exploits is a significantly better risk factor; (c) fixing in response to exploit presence in black markets yields the largest risk reduction.\n          <\/jats:p>","DOI":"10.1145\/2630069","type":"journal-article","created":{"date-parts":[[2014,8,12]],"date-time":"2014-08-12T13:53:48Z","timestamp":1407851628000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":152,"title":["Comparing Vulnerability Severity and Exploits Using Case-Control Studies"],"prefix":"10.1145","volume":"17","author":[{"given":"Luca","family":"Allodi","sequence":"first","affiliation":[{"name":"University of Trento, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabio","family":"Massacci","sequence":"additional","affiliation":[{"name":"University of Trento, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2014,8,15]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2008.916872"},{"key":"e_1_2_2_2_1","volume-title":"Proceedings of the 6th Workshop on Cybersecurity Security and Test.","author":"Allodi Luca","year":"2013","unstructured":"Luca Allodi , Vadim Kotov , and Fabio Massacci . 2013 . MalwareLab: Experimentation with Cybercrime attack tools . In Proceedings of the 6th Workshop on Cybersecurity Security and Test. Luca Allodi, Vadim Kotov, and Fabio Massacci. 2013. MalwareLab: Experimentation with Cybercrime attack tools. In Proceedings of the 6th Workshop on Cybersecurity Security and Test."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2382416.2382427"},{"key":"e_1_2_2_4_1","article-title":"Multiple significance tests: The Bonferroni method","volume":"310","author":"Martin Bland J.","year":"1995","unstructured":"J. Martin Bland and Douglas G. Altman . 1995 . Multiple significance tests: The Bonferroni method . Brit. Med. J. 310 , 6973 (1995), 170. J. Martin Bland and Douglas G. Altman. 1995. Multiple significance tests: The Bonferroni method. Brit. Med. J. 310, 6973 (1995), 170.","journal-title":"Brit. Med. J."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835804.1835821"},{"key":"e_1_2_2_6_1","unstructured":"Steve Christey and Brian Martin. 2013. Buying into the bias: Why vulnerability statistics suck. https:\/\/www.blackhat.com\/us-13\/archives.html&num;Martin.  Steve Christey and Brian Martin. 2013. Buying into the bias: Why vulnerability statistics suck. https:\/\/www.blackhat.com\/us-13\/archives.html&num;Martin."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1920261.1920299"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.2.4682.739"},{"key":"e_1_2_2_9_1","volume-title":"Proceeding of the USENIX Workshop on Large-Scale Exploits and Emergent Threats (LEET'12)","author":"Dumitras Tudor","year":"2012","unstructured":"Tudor Dumitras and Petros Efstathopoulos . 2012 . Ask WINE: Are we safer today&quest; Evaluating operating system security through big data analysis . In Proceeding of the USENIX Workshop on Large-Scale Exploits and Emergent Threats (LEET'12) . 11--11. Tudor Dumitras and Petros Efstathopoulos. 2012. Ask WINE: Are we safer today&quest; Evaluating operating system security through big data analysis. In Proceeding of the USENIX Workshop on Large-Scale Exploits and Emergent Threats (LEET'12). 11--11."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978672.1978683"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/0001-4575(86)90007-2"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1162666.1162671"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/NTMS.2011.5720656"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2009.36"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2382196.2382283"},{"key":"e_1_2_2_16_1","doi-asserted-by":"crossref","unstructured":"C. Herley and D. Florencio. 2010. Nobody sells gold for the price of silver: Dishonesty uncertainty and the underground economy. In Economics of Information Security and Privacy. Springer 33--53.  C. Herley and D. Florencio. 2010. Nobody sells gold for the price of silver: Dishonesty uncertainty and the underground economy. In Economics of Information Security and Privacy. Springer 33--53.","DOI":"10.1007\/978-1-4419-6967-5_3"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2009.08.023"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-36563-8_13"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2010.60"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.5555\/1946341.1946361"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2414456.2414459"},{"key":"e_1_2_2_23_1","volume-title":"Proceedings of the 6th Workshop on Economics and Information Security.","author":"Miller C.","year":"2007","unstructured":"C. Miller . 2007 . The legitimate vulnerability market: Inside the secretive world of 0-day exploit sales . In Proceedings of the 6th Workshop on Economics and Information Security. C. Miller. 2007. The legitimate vulnerability market: Inside the secretive world of 0-day exploit sales. In Proceedings of the 6th Workshop on Economics and Information Security."},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1315245.1315311"},{"key":"e_1_2_2_25_1","volume-title":"Proceedings of the 4th Workshop on Economics and Information Security.","author":"Ozment A.","year":"2005","unstructured":"A. Ozment . 2005 . The likelihood of vulnerability rediscovery and the social utility of vulnerability hunting . In Proceedings of the 4th Workshop on Economics and Information Security. A. Ozment. 2005. The likelihood of vulnerability rediscovery and the social utility of vulnerability hunting. In Proceedings of the 4th Workshop on Economics and Information Security."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1314257.1314261"},{"key":"e_1_2_2_27_1","volume-title":"Version 2.0.","author":"PCI Council","year":"2010","unstructured":"PCI Council . 2010. PCI DSS Requirements and Security Assessment Procedures , Version 2.0. ( 2010 ). https:\/\/www.pcisecuritystandards.org\/documents\/pci_dss_v2.pdf. PCI Council. 2010. PCI DSS Requirements and Security Assessment Procedures, Version 2.0. (2010). https:\/\/www.pcisecuritystandards.org\/documents\/pci_dss_v2.pdf."},{"key":"e_1_2_2_28_1","volume-title":"Johnson","author":"Quinn Stephen D.","year":"2010","unstructured":"Stephen D. Quinn , Karen A. Scarfone , Matthew Barrett , and Christopher S . Johnson . 2010 . Guide to Adopting and Using the Security Content Automation Protocol (SCAP) Version 1.0. Technical Report, National Institute of Standards and Technology, U.S. Department of Commerce , Special Publication 800-117. Stephen D. Quinn, Karen A. Scarfone, Matthew Barrett, and Christopher S. Johnson. 2010. Guide to Adopting and Using the Security Content Automation Protocol (SCAP) Version 1.0. Technical Report, National Institute of Standards and Technology, U.S. Department of Commerce, Special Publication 800-117."},{"key":"e_1_2_2_29_1","volume-title":"R: A Language and Environment for Statistical Computing","author":"Team R Core","year":"2012","unstructured":"R Core Team . 2012 . R: A Language and Environment for Statistical Computing . R Foundation for Statistical Computing, Vienna, Austria . http:\/\/www.R-project.org ISBN 3-900051-07-0. R Core Team. 2012. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. http:\/\/www.R-project.org ISBN 3-900051-07-0."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2009.5314220"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/IMF.2009.15"},{"key":"e_1_2_2_32_1","volume-title":"Muhammad Zubair Shafiq, and Alex X. Liu","author":"Shahzad Muhammad","year":"2012","unstructured":"Muhammad Shahzad , Muhammad Zubair Shafiq, and Alex X. Liu . 2012 . A large scale exploratory analysis of software vulnerability life cycles. In Proceedings of the 34th International Conference on Software Engineering. IEEE Press , 771--781. Muhammad Shahzad, Muhammad Zubair Shafiq, and Alex X. Liu. 2012. A large scale exploratory analysis of software vulnerability life cycles. In Proceedings of the 34th International Conference on Software Engineering. IEEE Press, 771--781."},{"key":"e_1_2_2_33_1","first-page":"25","article-title":"Can traditional fault prediction models be used for vulnerability prediction&quest; Empirical","volume":"18","author":"Shin Yonghee","year":"2013","unstructured":"Yonghee Shin and Laurie Williams . 2013 . Can traditional fault prediction models be used for vulnerability prediction&quest; Empirical Softw. Eng. 18 , 1 (2013), 25 -- 59 . DOI: http:\/\/dx.doi.org\/10.1007\/s10664-011-9190-8. 10.1007\/s10664-011-9190-8 Yonghee Shin and Laurie Williams. 2013. Can traditional fault prediction models be used for vulnerability prediction&quest; Empirical Softw. Eng. 18, 1 (2013), 25--59. DOI: http:\/\/dx.doi.org\/10.1007\/s10664-011-9190-8.","journal-title":"Softw. Eng."},{"key":"e_1_2_2_34_1","unstructured":"Symantec. 2011. Analysis of Malicious Web Activity by Attack Toolkits (online ed.). Symantec. http:\/\/www.symantec.com\/threatreport\/topic.jsp&quest;id=threat_activity_trends&aid=analysis_of_malicious_web _activity. (Last accessed June 1012).  Symantec. 2011. Analysis of Malicious Web Activity by Attack Toolkits (online ed.). Symantec. http:\/\/www.symantec.com\/threatreport\/topic.jsp&quest;id=threat_activity_trends&aid=analysis_of_malicious_web _activity. (Last accessed June 1012)."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-70567-3_22"}],"container-title":["ACM Transactions on Information and System Security"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2630069","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2630069","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T07:19:36Z","timestamp":1750231176000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2630069"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,8]]},"references-count":34,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2014,8]]}},"alternative-id":["10.1145\/2630069"],"URL":"https:\/\/doi.org\/10.1145\/2630069","relation":{},"ISSN":["1094-9224","1557-7406"],"issn-type":[{"value":"1094-9224","type":"print"},{"value":"1557-7406","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,8]]},"assertion":[{"value":"2013-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-08-15","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}