{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T16:21:10Z","timestamp":1783009270565,"version":"3.54.5"},"reference-count":68,"publisher":"Association for Computing Machinery (ACM)","issue":"ISSTA","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2025,6,22]]},"abstract":"<jats:p>As Ethereum continues to thrive, the Ethereum Virtual Machine (EVM) has become the cornerstone powering tens of millions of active smart contracts. Intuitively, security issues in EVMs could lead to inconsistent behaviors among smart contracts or even denial-of-service of the entire blockchain network. However, to the best of our knowledge, only a limited number of studies focus on the security of EVMs. Moreover, they suffer from 1) insufficient test input diversity and invalid semantics; and 2) the inability to automatically identify bugs and locate root causes.  \nTo bridge this gap, we propose OpDiffer, a differential testing framework for EVM, which takes advantage of LLMs and static analysis methods to address the above two limitations.  \nWe conducted the largest-scale evaluation, covering nine EVMs and uncovering 26 previously unknown bugs, 22 of which have been confirmed by developers and three have been assigned CNVD IDs. Compared to state-of-the-art baselines, OpDiffer can improve code coverage by at most 71.06%, 148.40% and 655.56%, respectively. Through an analysis of real-world deployed Ethereum contracts, we estimate that 7.21% of the contracts could trigger our identified EVM bugs under certain environmental settings, potentially resulting in severe negative impact on the Ethereum ecosystem.<\/jats:p>","DOI":"10.1145\/3728946","type":"journal-article","created":{"date-parts":[[2025,6,22]],"date-time":"2025-06-22T10:52:56Z","timestamp":1750589576000},"page":"1559-1582","source":"Crossref","is-referenced-by-count":4,"title":["OpDiffer: LLM-Assisted Opcode-Level Differential Testing of Ethereum Virtual Machine"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-6311-520X","authenticated-orcid":false,"given":"Jie","family":"Ma","sequence":"first","affiliation":[{"name":"Beihang University, Beijing, China"},{"name":"Zhongguancun Laboratory, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9980-7298","authenticated-orcid":false,"given":"Ningyu","family":"He","sequence":"additional","affiliation":[{"name":"The Hong Kong Polytechnic University, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7504-3457","authenticated-orcid":false,"given":"Jinwen","family":"Xi","sequence":"additional","affiliation":[{"name":"Zhongguancun Laboratory, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2065-9852","authenticated-orcid":false,"given":"Mingzhe","family":"Xing","sequence":"additional","affiliation":[{"name":"Zhongguancun Laboratory, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1100-8633","authenticated-orcid":false,"given":"Haoyu","family":"Wang","sequence":"additional","affiliation":[{"name":"Huazhong University of Science and Technology, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8992-651X","authenticated-orcid":false,"given":"Ying","family":"Gao","sequence":"additional","affiliation":[{"name":"Beihang University, Beijing, China"},{"name":"Zhongguancun Laboratory, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8417-2234","authenticated-orcid":false,"given":"Yinliang","family":"Yue","sequence":"additional","affiliation":[{"name":"Zhongguancun Laboratory, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,6,22]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Blue Alloy. 2025. Github revm repository. https:\/\/github.com\/bluealloy\/revm"},{"key":"e_1_2_1_2_1","unstructured":"Ether Alpha. 2025. Ethereum Client Diversity. https:\/\/clientdiversity.org"},{"key":"e_1_2_1_3_1","unstructured":"Alex Beregszaszi Pawe\u0142 Bylica Andrei Maiboroda and Matt Garnett. 2021. EIP-3540: EOF - EVM Object Format v1. https:\/\/eips.ethereum.org\/EIPS\/eip-3540"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3548606.3560624"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2014.15"},{"key":"e_1_2_1_6_1","volume-title":"Ethereum white paper. GitHub repository, 1","author":"Buterin Vitalik","year":"2013","unstructured":"Vitalik Buterin. 2013. Ethereum white paper. GitHub repository, 1 (2013), 22\u201323. https:\/\/ethereum.org\/en\/whitepaper"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3650212.3680358"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598110"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP46215.2023.10179386"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00127"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2908080.2908095"},{"key":"e_1_2_1_12_1","unstructured":"CoinMarketCap. 2025. Ethereum price today. https:\/\/coinmarketcap.com\/currencies\/ethereum"},{"key":"e_1_2_1_13_1","unstructured":"Ethereum community. 2025. Ethereum Improvement Proposals. https:\/\/ethereum.org\/en\/eips"},{"key":"e_1_2_1_14_1","unstructured":"Ethereum Javascript Community. 2025. Github ethereumjs repository. https:\/\/github.com\/ethereumjs\/ethereumjs-monorepo"},{"key":"e_1_2_1_15_1","unstructured":"Dan Mario Vega Mukul Kolpe Spencer Taylor-Brown and omahs. 2025. State Transition Tests Ethereum Execution Spec Tests. https:\/\/ethereum.github.io\/execution-spec-tests\/main\/tutorials\/state_transition"},{"key":"e_1_2_1_16_1","unstructured":"DappRadar. 2025. Top Ethereum Games. https:\/\/dappradar.com\/rankings\/protocol\/ethereum\/category\/games"},{"key":"e_1_2_1_17_1","first-page":"2021","volume":"202","unstructured":"National Vulnerability Database. 2021. CVE-2021-39137 Detail. https:\/\/nvd.nist.gov\/vuln\/detail\/CVE-2021-39137","journal-title":"National Vulnerability Database."},{"key":"e_1_2_1_18_1","unstructured":"Shihan Dou Haoxiang Jia Shenxi Wu Huiyuan Zheng Weikang Zhou Muling Wu Mingxu Chai Jessica Fan Caishuang Huang Yunbo Tao Yan Liu Enyu Zhou Ming Zhang Yuhao Zhou Yueming Wu Rui Zheng Ming Wen Rongxiang Weng Jingang Wang Xunliang Cai Tao Gui Xipeng Qiu Qi Zhang and Xuanjing Huang. 2024. What\u2019s Wrong with Your Code Generated by Large Language Models? An Extensive Study. arxiv:2407.06153. arxiv:2407.06153"},{"key":"e_1_2_1_19_1","unstructured":"Ethereum. 2025. Github evmone repository. https:\/\/github.com\/ethereum\/evmone"},{"key":"e_1_2_1_20_1","unstructured":"Ethereum. 2025. Github execution-specs repository. https:\/\/github.com\/ethereum\/execution-specs"},{"key":"e_1_2_1_21_1","unstructured":"Ethereum. 2025. Github Go Ethereum repository. https:\/\/github.com\/ethereum\/go-ethereum"},{"key":"e_1_2_1_22_1","unstructured":"Ethereum. 2025. Github Py-EVM repository. https:\/\/github.com\/ethereum\/py-evm"},{"key":"e_1_2_1_23_1","unstructured":"Ethereum.org. 2025. Decentralized finance (DeFi). https:\/\/ethereum.org\/en\/defi"},{"key":"e_1_2_1_24_1","unstructured":"Ethereum.org. 2025. Ethereum Virtual Machine (EVM) implementations. https:\/\/ethereum.org\/en\/developers\/docs\/evm"},{"key":"e_1_2_1_25_1","unstructured":"Ethereum.org. 2025. The history of Ethereum. https:\/\/ethereum.org\/en\/history"},{"key":"e_1_2_1_26_1","unstructured":"Ethereum.org. 2025. Non-fungible tokens (NFT). https:\/\/ethereum.org\/en\/nft"},{"key":"e_1_2_1_27_1","unstructured":"Etherscan. 2025. The Ethereum Blockchain Explorer. https:\/\/etherscan.io"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3341175"},{"key":"e_1_2_1_29_1","unstructured":"Google. 2025. Coverage profiling support for integration tests. https:\/\/go.dev\/doc\/build-cover"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00120"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3617850"},{"key":"e_1_2_1_32_1","volume-title":"EOSAFE: Security Analysis of EOSIO Smart Contracts. In 30th USENIX Security Symposium (USENIX Security 21)","author":"He Ningyu","year":"2021","unstructured":"Ningyu He, Ruiyi Zhang, Haoyu Wang, Lei Wu, Xiapu Luo, Yao Guo, Ting Yu, and Xuxian Jiang. 2021. EOSAFE: Security Analysis of EOSIO Smart Contracts. In 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 1271\u20131288. isbn:978-1-939133-24-3 https:\/\/www.usenix.org\/conference\/usenixsecurity21\/presentation\/he-ningyu"},{"key":"e_1_2_1_33_1","unstructured":"Hyperledger. 2025. Github Besu Ethereum Client repository. https:\/\/github.com\/hyperledger\/besu\/"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238177"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616251"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2021.23108"},{"key":"e_1_2_1_37_1","volume-title":"Scalpel: The Python Static Analysis Framework. arxiv:2202.11840. arxiv:2202.11840","author":"Li Li","year":"2022","unstructured":"Li Li, Jiawei Wang, and Haowei Quan. 2022. Scalpel: The Python Static Analysis Framework. arxiv:2202.11840. arxiv:2202.11840"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00089"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3576915.3623166"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639180"},{"key":"e_1_2_1_41_1","volume-title":"LOKI: State-Aware Fuzzing Framework for the Implementation of Blockchain Consensus Protocols. In 30th Annual Network and Distributed System Security Symposium, NDSS 2023","author":"Ma Fuchen","year":"2023","unstructured":"Fuchen Ma, Yuanliang Chen, Meng Ren, Yuanhang Zhou, Yu Jiang, Ting Chen, Huizhong Li, and Jiaguang Sun. 2023. LOKI: State-Aware Fuzzing Framework for the Implementation of Blockchain Consensus Protocols. In 30th Annual Network and Distributed System Security Symposium, NDSS 2023, San Diego, California, USA, February 27 - March 3, 2023. The Internet Society. https:\/\/www.ndss-symposium.org\/ndss-paper\/loki-state-aware-fuzzing-framework-for-the-implementation-of-blockchain-consensus-protocols\/"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","unstructured":"Jie Ma. 2025. OpDiffer: LLM-Assisted Opcode-Level Differential Testing of Ethereum Virtual Machine. https:\/\/doi.org\/10.5281\/zenodo.15195943 10.5281\/zenodo.15195943","DOI":"10.5281\/zenodo.15195943"},{"key":"e_1_2_1_43_1","volume-title":"Abusing the Ethereum Smart Contract Verification Services for Fun and Profit. In 31st Annual Network and Distributed System Security Symposium, NDSS 2024","author":"Ma Pengxiang","year":"2024","unstructured":"Pengxiang Ma, Ningyu He, Yuhua Huang, Haoyu Wang, and Xiapu Luo. 2024. Abusing the Ethereum Smart Contract Verification Services for Fun and Profit. In 31st Annual Network and Distributed System Security Symposium, NDSS 2024, San Diego, California, USA, February 26 - March 1, 2024. The Internet Society. https:\/\/www.ndss-symposium.org\/ndss-paper\/abusing-the-ethereum-smart-contract-verification-services-for-fun-and-profit\/"},{"key":"e_1_2_1_44_1","volume-title":"Proceedings of the 37th International Conference on Neural Information Processing Systems (NIPS \u201923)","author":"Madaan Aman","year":"2023","unstructured":"Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. SELF-REFINE: iterative refinement with self-feedback. In Proceedings of the 37th International Conference on Neural Information Processing Systems (NIPS \u201923). Curran Associates Inc., Red Hook, NY, USA. Article 2019, 61 pages."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3503921.3503923"},{"key":"e_1_2_1_46_1","unstructured":"Marius van der Wijden Martin Holst Swende. 2020. EIP-3155: EVM trace specification [DRAFT]. https:\/\/eips.ethereum.org\/EIPS\/eip-3155"},{"key":"e_1_2_1_47_1","unstructured":"NethermindEth. 2025. Github Nethermind Ethereum client repository. https:\/\/github.com\/NethermindEth\/nethermind"},{"key":"e_1_2_1_48_1","unstructured":"Beijing Academy of Blockchain and Edge Computing. 2025. chainmaker document. https:\/\/docs.chainmaker.org.cn"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.27"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP54263.2024.00137"},{"key":"e_1_2_1_51_1","unstructured":"SealSC. 2025. Github SealEVM repository. https:\/\/github.com\/SealSC\/SealEVM"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598059"},{"key":"e_1_2_1_53_1","unstructured":"smlXL Inc.. 2025. An Ethereum Virtual Machine Opcodes Interactive Reference. https:\/\/www.evm.codes\/?fork=cancun"},{"key":"e_1_2_1_54_1","volume-title":"33rd USENIX Security Symposium (USENIX Security 24)","author":"Sun Tianle","year":"2024","unstructured":"Tianle Sun, Ningyu He, Jiang Xiao, Yinliang Yue, Xiapu Luo, and Haoyu Wang. 2024. All Your Tokens are Belong to Us: Demystifying Address Verification Vulnerabilities in Solidity Smart Contracts. In 33rd USENIX Security Symposium (USENIX Security 24). USENIX Association, Philadelphia, PA. 3567\u20133584. isbn:978-1-939133-44-1 https:\/\/www.usenix.org\/conference\/usenixsecurity24\/presentation\/sun-tianle"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639117"},{"key":"e_1_2_1_56_1","unstructured":"Martin Holst Swende. 2025. Github Go evmlab repository. https:\/\/github.com\/holiman\/goevmlab"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSP51992.2021.00018"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3243780"},{"key":"e_1_2_1_59_1","unstructured":"Marius van der Wijden. 2025. Github FuzzyVM repository. https:\/\/github.com\/MariusVanDerWijden\/FuzzyVM"},{"key":"e_1_2_1_60_1","unstructured":"Sam Wilson. 2023. Ethereum Execution Layer Specification. https:\/\/blog.ethereum.org\/2023\/08\/29\/eel-spec"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3586053"},{"key":"e_1_2_1_62_1","volume-title":"Ethereum: A secure decentralised generalised transaction ledger. Ethereum project yellow paper, 151","author":"Wood Gavin","year":"2014","unstructured":"Gavin Wood. 2014. Ethereum: A secure decentralised generalised transaction ledger. Ethereum project yellow paper, 151, 2014 (2014), 1\u201332."},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639152"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3650212.3680388"},{"key":"e_1_2_1_65_1","volume-title":"15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21)","author":"Yang Youngseok","year":"2021","unstructured":"Youngseok Yang, Taesoo Kim, and Byung-Gon Chun. 2021. Finding Consensus Bugs in Ethereum via Multi-transaction Differential Fuzzing. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21). USENIX Association, 349\u2013365. isbn:978-1-939133-22-9 https:\/\/www.usenix.org\/conference\/osdi21\/presentation\/yang"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP54263.2024.00146"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639140"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00188"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3728946","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,16]],"date-time":"2025-07-16T16:50:07Z","timestamp":1752684607000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3728946"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,22]]},"references-count":68,"journal-issue":{"issue":"ISSTA","published-print":{"date-parts":[[2025,6,22]]}},"alternative-id":["10.1145\/3728946"],"URL":"https:\/\/doi.org\/10.1145\/3728946","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,22]]}}}