{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T19:25:37Z","timestamp":1774466737256,"version":"3.50.1"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,2,2]],"date-time":"2024-02-02T00:00:00Z","timestamp":1706832000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,2,2]],"date-time":"2024-02-02T00:00:00Z","timestamp":1706832000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Funda\u00e7\u00e3o para a C\u00eeencia e a Tecnologia","award":["UIDB\/50021\/2020"],"award-info":[{"award-number":["UIDB\/50021\/2020"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2024,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Blockchain programs (also known as smart contracts) manage valuable assets like cryptocurrencies and tokens, and implement protocols in domains like decentralized finance (DeFi) and supply-chain management. These types of applications require a high level of security that is hard to achieve due to the transparency of public blockchains. Numerous tools support developers and auditors in the task of detecting weaknesses. As a young technology, blockchains and utilities evolve fast, making it challenging for tools and developers to keep up with the pace. In this work, we study the robustness of code analysis tools and the evolution of weakness detection on a dataset representing six years of blockchain activity. We focus on Ethereum as the crypto ecosystem with the largest number of developers and deployed programs. We investigate the behavior of single tools as well as the agreement of several tools addressing similar weaknesses. Our study is the first that is based on the entire body of deployed bytecode on Ethereum\u2019s main chain. We achieve this coverage by considering bytecodes as equivalent if they share the same skeleton. The skeleton of a bytecode is obtained by omitting functionally irrelevant parts. This reduces the 48 million contracts deployed on Ethereum up to January 2022 to 248\u00a0328 contracts with distinct skeletons. For bulk execution, we utilize the open-source framework SmartBugs that facilitates the analysis of Solidity smart contracts, and enhance it to accept also bytecode as the only input. Moreover, we integrate six further tools for bytecode analysis. The execution of the 12 tools included in our study on the dataset took 30 CPU years. While the tools report a total of 1\u00a0307\u00a0486 potential weaknesses, we observe a decrease in reported weaknesses over time, as well as a degradation of tools to varying degrees.<\/jats:p>","DOI":"10.1007\/s10664-023-10414-8","type":"journal-article","created":{"date-parts":[[2024,2,2]],"date-time":"2024-02-02T08:03:30Z","timestamp":1706861010000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Evolution of automated weakness detection in Ethereum bytecode: a comprehensive study"],"prefix":"10.1007","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4217-4530","authenticated-orcid":false,"given":"Monika","family":"di\u00a0Angelo","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1996-6134","authenticated-orcid":false,"given":"Thomas","family":"Durieux","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6612-9013","authenticated-orcid":false,"given":"Jo\u00e3o\u00a0F.","family":"Ferreira","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8950-1551","authenticated-orcid":false,"given":"Gernot","family":"Salzer","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,2,2]]},"reference":[{"key":"10414_CR1","doi-asserted-by":"publisher","DOI":"10.1109\/DAPPCON.2019.00018","author":"M di Angelo","year":"2019","unstructured":"di Angelo M, Salzer G (2019) A Survey of Tools for Analyzing Ethereum Smart Contracts, IEEE international conference on decentralized applications and infrastructures (DAPPCON), pp 69\u201378. Piscataway, NJ, USA. https:\/\/doi.org\/10.1109\/DAPPCON.2019.00018","journal-title":"Piscataway, NJ, USA"},{"key":"10414_CR2","doi-asserted-by":"publisher","unstructured":"di Angelo M, Salzer G (2024) Consolidation of ground truth sets for weakness detection in smart contracts. In: Essex A, Matsuo S, Kulyk O, Gudgeon L, Klages-Mundt A, Perez D, Werner S, Bracciali A, Goodell G (eds) Financial Cryptography and Data Security. FC 2023 International Workshops, Springer, LNCS, pp 439\u2013455, https:\/\/doi.org\/10.1007\/978-3-031-48806-1_28","DOI":"10.1007\/978-3-031-48806-1_28"},{"key":"10414_CR3","doi-asserted-by":"publisher","unstructured":"Brent L, Jurisevic A, Kong M, Liu E, Gauthier F, Gramoli V, Holz R, Scholz B (2018) Vandal: A Scalable Security Analysis Framework for Smart Contracts. arXiv https:\/\/doi.org\/10.48550\/arXiv.1809.03981","DOI":"10.48550\/arXiv.1809.03981"},{"key":"10414_CR4","doi-asserted-by":"publisher","unstructured":"Brent L, Grech N, Lagouvardos S, Scholz B, Smaragdakis Y (2020) Ethainter: a smart contract security analyzer for composite vulnerabilities, Association for Computing Machinery. In: Proceedings of the 41st ACM SIGPLAN conference on programming language design and implementation, London UK, PLDI 2020 16:454\u2013469, New York, NY, USA. https:\/\/doi.org\/10.1145\/3385412.3385990","DOI":"10.1145\/3385412.3385990"},{"issue":"3","key":"10414_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3391195","volume":"53","author":"H Chen","year":"2020","unstructured":"Chen H, Pendleton M, Njilla L, Xu S (2020) A Survey on Ethereum Systems Security. ACM Comput Surv 53(3):1\u201343. https:\/\/doi.org\/10.1145\/3391195","journal-title":"ACM Comput Surv"},{"key":"10414_CR6","doi-asserted-by":"publisher","DOI":"10.1109\/PRDC53464.2021.00013","author":"B Dias","year":"2021","unstructured":"Dias B, Ivaki N, Laranjeiro N (2021) An Empirical Evaluation of the Effectiveness of Smart Contract Verification Tools, IEEE 26th Pacific Rim International Symposium on Dependable Computing (PRDC), p 17\u201326. IEEE. https:\/\/doi.org\/10.1109\/PRDC53464.2021.00013","journal-title":"IEEE"},{"key":"10414_CR7","unstructured":"Dika A (2017) Ethereum Smart Contracts: Security Vulnerabilities and Security Tools. NTNU,"},{"key":"10414_CR8","doi-asserted-by":"publisher","unstructured":"Durieux T, Ferreira JF, Abreu R, Cruz P (2020) Empirical review of automated analysis tools on 47,587 Ethereum smart contracts. Proceedings of the ACM\/IEEE 42nd International Conference on Software Engineering, New York, NY, USA. p 530\u2013541. ACM https:\/\/doi.org\/10.1145\/3377811.3380364,","DOI":"10.1145\/3377811.3380364"},{"key":"10414_CR9","doi-asserted-by":"publisher","unstructured":"Ferreira JF, Cruz P, Durieux T, Abreu R (2020) Smartbugs: A framework to analyze solidity smart contracts. In: Proceedings of the 35th IEEE\/ACM International Conference on Automated Software Engineering, p 1349\u20131352, ACM, New York, NY, USA, https:\/\/doi.org\/10.1145\/3324884.3415298,","DOI":"10.1145\/3324884.3415298"},{"key":"10414_CR10","doi-asserted-by":"publisher","unstructured":"Ferreira Torres C, Sch\u00fctte J, State R (2018) Osiris: Hunting for Integer Bugs in Ethereum Smart Contracts. Proceedings of the 34th Annual Computer Security Applications Conference, pp 664\u2013676, New York, NY, USA https:\/\/doi.org\/10.1145\/3274694.3274737,","DOI":"10.1145\/3274694.3274737"},{"key":"10414_CR11","doi-asserted-by":"publisher","unstructured":"Ghaleb A, Pattabiraman K (2020) How effective are smart contract analysis tools? Evaluating smart contract static analysis tools using bug injection. In: Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, ACM New York, NY, USA, pp 415\u2013427. https:\/\/doi.org\/10.1145\/3395363.3397385","DOI":"10.1145\/3395363.3397385"},{"key":"10414_CR12","doi-asserted-by":"publisher","unstructured":"Grech N, Kong M, Jurisevic A, Brent L, Scholz B, Smaragdakis Y (2018) MadMax: Surviving out-of-gas conditions in Ethereum smart contracts. Proceedings of the ACM on Programming Languages, ACM New York, NY, USA, 2(OOPSLA):1\u201327. https:\/\/doi.org\/10.1145\/3276486,","DOI":"10.1145\/3276486"},{"key":"10414_CR13","unstructured":"Gupta BC (2019) Analysis of Ethereum Smart Contracts - A Security Perspective. Indian Institute of Technology Kanpur"},{"key":"10414_CR14","doi-asserted-by":"crossref","unstructured":"Gupta BC, Kumar N, Handa A, Shukla SK (2020) An Insecurity Study of Ethereum Smart Contracts. In: Batina L Picek S Mondal M (eds) Security Privacy, Cryptography Applied . Springer International Publishing, Cham, Engineering, pp 188\u2013207","DOI":"10.1007\/978-3-030-66626-2_10"},{"key":"10414_CR15","doi-asserted-by":"publisher","unstructured":"Ji S, Kim D, Im H (2021) Evaluating Countermeasures for Verifying the Integrity of Ethereum Smart Contract Applications. IEEE Access, 9:90029\u201390042, IEEE https:\/\/doi.org\/10.1109\/ACCESS.2021.3091317,","DOI":"10.1109\/ACCESS.2021.3091317"},{"key":"10414_CR16","unstructured":"Krupp J, Rossow C, (2018) teEther: Gnawing at Ethereum to Automatically Exploit Smart Contracts. In: 27th USENIX conference on security symposium (USENIX Security 18), Baltimore, MD USENIX Association, (18):1317\u20131333. https:\/\/www.usenix.org\/conference\/usenixsecurity18\/presentation\/krupp"},{"key":"10414_CR17","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2022.3169902","author":"SS Kushwaha","year":"2022","unstructured":"Kushwaha SS, Joshi S, Singh D, Kaur M, Lee H-N (2022) Ethereum Smart Contract Analysis Tools: A Systematic Review. IEEE Access. https:\/\/doi.org\/10.1109\/ACCESS.2022.3169902","journal-title":"IEEE Access"},{"key":"10414_CR18","doi-asserted-by":"publisher","first-page":"6605","DOI":"10.1109\/ACCESS.2021.3140091","volume":"10","author":"SS Kushwaha","year":"2022","unstructured":"Kushwaha SS, Joshi S, Singh D, Kaur M, Lee H-N (2022) Systematic Review of Security Vulnerabilities in Ethereum Blockchain Smart Contract. IEEE Access 10:6605\u20136621. https:\/\/doi.org\/10.1109\/ACCESS.2021.3140091","journal-title":"IEEE Access"},{"key":"10414_CR19","doi-asserted-by":"publisher","unstructured":"Leid A, van der Merwe B, Visser W (2020) Testing Ethereum Smart Contracts: A Comparison of Symbolic Analysis and Fuzz Testing Tools. In: Conference of the South African Institute of Computer Scientists and Information Technologists 2020. ACM New York, NY, USA, pp 35\u201343. https:\/\/doi.org\/10.1145\/3410886.3410907,","DOI":"10.1145\/3410886.3410907"},{"issue":"2","key":"10414_CR20","doi-asserted-by":"publisher","first-page":"203","DOI":"10.3390\/e22020203","volume":"22","author":"A L\u00f3pez Vivar","year":"2020","unstructured":"L\u00f3pez Vivar A, Castedo AT, Sandoval Orozco AL, Garc\u00eda Villalba LJ (2020) An analysis of smart contracts security threats alongside existing solutions. Entropy 22(2):203. https:\/\/doi.org\/10.3390\/e22020203","journal-title":"Entropy"},{"key":"10414_CR21","doi-asserted-by":"publisher","unstructured":"Luu L, Chu D-H, Olickel H, Saxena P, Hobor A (2016) Making smart contracts smarter. In: Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, ACM New York, NY, USA, pp 254\u2013269 https:\/\/doi.org\/10.1145\/2976749.2978309,","DOI":"10.1145\/2976749.2978309"},{"key":"10414_CR22","unstructured":"Mueller B (2018) Smashing ethereum smart contracts for fun and real profit. 9th Annual HITB Security Conference (HITBSecConf). Amsterdam, Netherlands HITB, https:\/\/raw.githubusercontent.com\/b-mueller\/smashing-smart-contracts\/master\/smashing-smart-contracts-1of1.pdf,"},{"key":"10414_CR23","doi-asserted-by":"publisher","unstructured":"Nikoli\u0107 I, Kolluri A, Sergey I, Saxena P, Hobor A (2018) Finding the greedy, prodigal, and suicidal contracts at scale. In: Proceedings of the 34th annual computer security applications conference. New York, NY, USA ACM. pp 653\u2013663. https:\/\/doi.org\/10.1145\/3274694.3274743,","DOI":"10.1145\/3274694.3274743"},{"key":"10414_CR24","unstructured":"Parizi RM, Dehghantanha A, Choo Kim-Kwang R, Singh A (2018) Empirical vulnerability analysis of automated smart contracts security testing on blockchains. In: Proceedings of the 28th annual international conference on computer science and software engineering. vol 18 pp 103\u2013113, IBM Corp. http:\/\/dl.acm.org\/citation.cfm?id=3291291.3291303,"},{"key":"10414_CR25","doi-asserted-by":"publisher","unstructured":"Rameder H, di Angelo M, Salzer G (2022) Review of automated vulnerability analysis of smart contracts on ethereum. Front Blockchain 5. https:\/\/doi.org\/10.3389\/fbloc.2022.814977","DOI":"10.3389\/fbloc.2022.814977"},{"key":"10414_CR26","doi-asserted-by":"publisher","unstructured":"Ren M, Yin Z, Ma F, Xu Z, Jiang Y, Sun C, Li H, Cai Y (2021) Empirical evaluation of smart contract testing: what is the best choice? In: Proceedings of the 30th ACM SIGSOFT international symposium on software testing and analysis. pp 566\u2013579. ACM New York, NY, USA. https:\/\/doi.org\/10.1145\/3460319.3464837","DOI":"10.1145\/3460319.3464837"},{"key":"10414_CR27","doi-asserted-by":"publisher","unstructured":"Schneidewind C, Grishchenko I, Scherer M, Maffei M (2020) EThor: practical and provably sound static analysis of ethereum smart contracts. Proceedings of the 2020 ACM SIGSAC conference on computer and communications security. Association for Computing Machinery, New York, NY, USA. pp 621\u2013640. https:\/\/doi.org\/10.1145\/3372297.3417250","DOI":"10.1145\/3372297.3417250"},{"key":"10414_CR28","doi-asserted-by":"publisher","unstructured":"Tang X, Zhou K, Cheng J, Li H, Yuan Y (2021) The vulnerabilities in smart contracts: a survey. In: Sun X, Zhang X, Xia Z, Bertino E (eds) International conference on artificial intelligence and security (ICAIS). Communications in computer and information science, vol CCIS 1424, Springer, Cham, pp 177\u2013190. https:\/\/doi.org\/10.1007\/978-3-030-78621-2_14","DOI":"10.1007\/978-3-030-78621-2_14"},{"issue":"7","key":"10414_CR29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3464421","volume":"54","author":"P Tolmach","year":"2022","unstructured":"Tolmach P, Li Y, Lin S-W, Liu Y, Li Z (2022) A survey of smart contract formal specification and verification. ACM Comput Surv 54(7):1\u201338. https:\/\/doi.org\/10.1145\/3464421","journal-title":"ACM Comput Surv"},{"key":"10414_CR30","doi-asserted-by":"publisher","unstructured":"Tsankov P, Dan A, Drachsler-Cohen D, Gervais A, B\u00fcnzli F, Vechev M (2018) Securify: practical security analysis of smart contracts. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. pp 67\u201382. ACM New York, NY, USA. https:\/\/doi.org\/10.1145\/3243734.3243780","DOI":"10.1145\/3243734.3243780"},{"issue":"2","key":"10414_CR31","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-020-9284-9","volume":"15","author":"Z Wang","year":"2021","unstructured":"Wang Z, Jin H, Dai W, Choo K-KR, Zou D (2021) Ethereum smart contract security research: survey and future research opportunities. Front Comput Sci 15(2):152802. https:\/\/doi.org\/10.1007\/s11704-020-9284-9","journal-title":"Front Comput Sci"},{"key":"10414_CR32","doi-asserted-by":"publisher","unstructured":"Zhang P, Xiao F, Luo X (2020) A framework and dataset for bugs in ethereum smart contracts. In: 2020 IEEE international conference on software maintenance and evolution (ICSME), pp 139\u2013150. https:\/\/doi.org\/10.1109\/ICSME46990.2020.00023","DOI":"10.1109\/ICSME46990.2020.00023"},{"key":"10414_CR33","doi-asserted-by":"publisher","unstructured":"Zhou H, Milani Fard A, Makanju A (2022) The State of Ethereum Smart Contracts Security: Vulnerabilities, Countermeasures, and Tool Support. J Cybersec Priv, 2(2):358\u2013378. Multidisciplinary Digital Publishing Institute, https:\/\/doi.org\/10.3390\/jcp2020019","DOI":"10.3390\/jcp2020019"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10414-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-023-10414-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10414-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,23]],"date-time":"2024-03-23T02:18:18Z","timestamp":1711160298000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-023-10414-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,2]]},"references-count":33,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,3]]}},"alternative-id":["10414"],"URL":"https:\/\/doi.org\/10.1007\/s10664-023-10414-8","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,2]]},"assertion":[{"value":"18 October 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 February 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of interest"}}],"article-number":"41"}}