{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T22:13:03Z","timestamp":1788300783233,"version":"build-2803163510"},"reference-count":40,"publisher":"Association for Computing Machinery (ACM)","issue":"9","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Commun. ACM"],"published-print":{"date-parts":[[2026,9,1]]},"abstract":"<jats:p>Test or prove? These two approaches to software verification have long been presented as opposites. One is dynamic, the other static: A test executes the program, a proof only analyzes the program text. A different perspective is emerging, in which testing and proving are complementary rather than competing techniques for producing software of verified quality.<\/jats:p>\n                  <jats:p>\n                    Work performed over the past few years and reviewed here develops this complementarity by taking advantage of \"design by contract,\" as available in Eiffel, and exploiting a feature of modern program-proving tools based on \"satisfiability modulo theories\" (SMT):\n                    <jats:italic toggle=\"yes\">counterexample<\/jats:italic>\n                    generation. A counterexample is an input combination that makes the program fail. If we are trying to prove a program correct, we hope not to find any. One can, however, apply counterexample generation to incorrect programs, as a tool for\n                    <jats:italic toggle=\"yes\">automatic test generation<\/jats:italic>\n                    . We can also introduce faults into a correct program and turn the counterexamples into an automatically generated\n                    <jats:italic toggle=\"yes\">regression test suite<\/jats:italic>\n                    with full coverage. Additionally, we can use these mechanisms to help produce\n                    <jats:italic toggle=\"yes\">program fixes<\/jats:italic>\n                    for incorrect programs, with a guarantee that the fixes are correct. All three applications, leveraging on the mechanisms of Eiffel and design by contract, hold significant promise to address some of the challenges of program testing, software maintenance, and automatic program repair.\n                  <\/jats:p>","DOI":"10.1145\/3793869","type":"journal-article","created":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T21:07:55Z","timestamp":1787778475000},"page":"76-85","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Combining Tests and Proofs with Contracts for Better Software Verification"],"prefix":"10.1145","volume":"69","author":[{"given":"Li","family":"Huang","sequence":"first","affiliation":[{"name":"Constructor Institute of Technology, Schaffhausen, SH, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5985-7434","authenticated-orcid":false,"given":"Bertrand","family":"Meyer","sequence":"additional","affiliation":[{"name":"ETH Zurich, Chair of Software Engineering, Zurich, ZH, Switzerland"},{"name":"Eiffel Software, Goleta, California, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manuel","family":"Oriol","sequence":"additional","affiliation":[{"name":"Constructor Institute of Technology, Schaffhausen, SH, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,9,1]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"crossref","unstructured":"Angeletti D. et al.\u00a0Improving the automatic test generation process for coverage analysis using cbmc. In: RCRA@ AI* IA (2009).","DOI":"10.1007\/978-3-642-04772-5_38"},{"key":"e_1_3_1_3_2","first-page":"14","article-title":"The SMT-LIB Standard: Version 2.0","volume":"13","author":"Barrett C.","year":"2010","unstructured":"Barrett, C., Stump, A., and Tinelli, C. et al.\u00a0The SMT-LIB Standard: Version 2.0. In: Intern. Workshop on Satisfiability Modulo Theories\u00a013\u00a0(2010), 14.","journal-title":"Intern. Workshop on Satisfiability Modulo Theories"},{"key":"e_1_3_1_4_2","doi-asserted-by":"crossref","unstructured":"Beyer D. et al.\u00a0Generating tests from counterexamples. In\u00a0Intern. Conf. on Software Engineering. IEEE\u00a0(2004) 326\u2013335.","DOI":"10.1109\/ICSE.2004.1317455"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/1241572.1241581"},{"key":"e_1_3_1_6_2","unstructured":"Brenguier R. Cordeiro L. Kroening D. and Schrammel P.\u00a0Jbmc: A bounded model checking tool for java bytecode. arXiv preprint (2023) arXiv:2302.02381."},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10009-020-00570-3"},{"key":"e_1_3_1_8_2","first-page":"404","article-title":"Better Counterexamples for Dafny","author":"Chakarov A.","year":"2022","unstructured":"Chakarov, A., Fedchin, A., Rakamari\u0107, Z., and Rungta, N.\u00a0Better Counterexamples for Dafny. In\u00a0Intern. Conf. on Tools and Algorithms for the Construction and Analysis of Systems. Springer\u00a0(2022), 404\u2013411.","journal-title":"Intern. Conf. on Tools and Algorithms for the Construction and Analysis of Systems"},{"key":"e_1_3_1_9_2","doi-asserted-by":"crossref","unstructured":"Ciupa I. Leitner A. Oriol M. and Meyer B.\u00a0Artoo: Adaptive random testing for object-oriented software. In\u00a0Proceedings of the 30th Intern. Conf. on Software Engineering\u00a0(2008) 71\u201380.","DOI":"10.1145\/1368088.1368099"},{"key":"e_1_3_1_10_2","doi-asserted-by":"crossref","unstructured":"Cok D.R.\u00a0JML and OpenJML for Java 16. In\u00a0Intern. Workshop on Formal Techniques for Java-like Programs (FTfJP). ACM\u00a0(2021) 65\u201367.","DOI":"10.1145\/3464971.3468417"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/359104.359106"},{"key":"e_1_3_1_12_2","first-page":"1","volume-title":"Structured Programming.","author":"Dijkstra E.W.","year":"1972","unstructured":"Dijkstra, E.W.: Notes on structured programming. In\u00a0Structured Programming. O.J. Dahl, E.W.\u00a0Dijkstra, and C.A.R.\u00a0Hoare,\u00a0(Eds.)\u00a0 Academic Press\u00a0(1972;\u00a0originally circulated in 1970),\u00a01\u201382."},{"key":"e_1_3_1_13_2","volume-title":"A Discipline of Programming","author":"Dijkstra E.W.","year":"1976","unstructured":"Dijkstra, E.W.\u00a0A Discipline of Programming. Prentice Hall (1976)."},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2007.01.015"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/318774.318939"},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","unstructured":"Goodenough J.B. and Gerhart S.L.\u00a0Towards a theory of test data selection. In\u00a0Proceedings of the Intern. Conf. on Reliable Software. ACM\u00a0(1975) 493\u2013510.","DOI":"10.1145\/800027.808473"},{"key":"e_1_3_1_17_2","doi-asserted-by":"crossref","unstructured":"Gorla D. Kumar S. Roselli Lorenzini P.N. and Alipourfaz A.: CubeTesterAI: Automated JUnit test generation using the LLaMA model. In\u00a02025 IEEE Conf. on Software Testing Verification and Validation. IEEE\u00a0\u00a0(Apr. 2025) 565\u2013576;\u00a0https:\/\/doi.ieeecomputersociety.org\/10.1109\/ICST62969.2025.10988978","DOI":"10.1109\/ICST62969.2025.10988978"},{"key":"e_1_3_1_18_2","unstructured":"Huang L. et al.\u00a0Lessons from formally verified deployed software systems. ACM Computing Surveys\u00a0to appear (2025) preprint (of extended version) available as arXiv:2301.02206"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1859"},{"key":"e_1_3_1_20_2","doi-asserted-by":"crossref","unstructured":"Huang L. Meyer B. Mustafin I. and Oriol M.\u00a0Execution-free program repair. In\u00a0Companion Proceedings of the 32nd ACM Intern. Conf. on the Foundations of Software Engineering\u00a0(2024) 517\u2013521.","DOI":"10.1145\/3663529.3663789"},{"key":"e_1_3_1_21_2","doi-asserted-by":"crossref","unstructured":"Huang L. Meyer B. and Oriol M.\u00a0Improving counterexample quality from failed program verification. In\u00a0Intern. Symp. on Software Reliability Engineering Workshops. IEEE\u00a0(2022) 268\u2013273.","DOI":"10.1109\/ISSREW55968.2022.00078"},{"key":"e_1_3_1_22_2","doi-asserted-by":"crossref","unstructured":"Huang L. Meyer B. and Oriol M.\u00a0Seeding contradiction: A fast method for generating full-coverage test suites. In\u00a0IFIP Intern. Conf. on Testing Software and Systems. Springer\u00a0(2023) 52\u201370.","DOI":"10.1007\/978-3-031-43240-8_4"},{"key":"e_1_3_1_23_2","doi-asserted-by":"crossref","unstructured":"Huang L. Meyer B. and Oriol M.\u00a0Is MCDC really better? Lessons from combining tests and proofs. In\u00a0Intern. Conf. on Tests and Proofs. Springer\u00a0(2024) 25\u201344.","DOI":"10.1007\/978-3-031-72044-4_2"},{"key":"e_1_3_1_24_2","doi-asserted-by":"crossref","unstructured":"Huang L. Meyer B. and Weber R.\u00a0Loop unrolling: Formal definition and application to testing. In\u00a0Proceedings of the Intern. Conf. on Testing Software and Systems. Springer Lecture Notes in Computer Science (2025) preprint available as arXiv:2502.15535","DOI":"10.1007\/978-3-032-05188-2_2"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.119.2.254"},{"key":"e_1_3_1_26_2","doi-asserted-by":"crossref","unstructured":"Kroening D. and Tautschnig M.\u00a0CBMC\u2013C bounded model checker (competition contribution). In\u00a0Tools and Algorithms for the Construction and Analysis of Systems: 20th Intern. Conf.\u00a0Springer\u00a0(2014) 389\u2013391.","DOI":"10.1007\/978-3-642-54862-8_26"},{"key":"e_1_3_1_27_2","doi-asserted-by":"crossref","unstructured":"Le Goues C. Leino K.R.M. and Moskal M.\u00a0The Boogie Verification Debugger. In\u00a0Intern. Conf. on Software Engineering and Formal Methods. Springer\u00a0(2011) 407\u2013414.","DOI":"10.1007\/978-3-642-24690-6_28"},{"key":"e_1_3_1_28_2","doi-asserted-by":"crossref","unstructured":"Ma L. et al.\u00a0SpecGen: Automated generation of formal program specifications via large language models. In\u00a02025 IEEE\/ACM 47th Intern. Conf. on Software Engineering (ICSE). IEEE\u00a0(May 2025) 16\u201328;\u00a0https:\/\/doi.ieeecomputersociety.org\/10.1109\/ICSE55347.2025.00129","DOI":"10.1109\/ICSE55347.2025.00129"},{"key":"e_1_3_1_29_2","volume-title":"Object-Oriented Software Construction, second edition","author":"Meyer B.","year":"1997","unstructured":"Meyer, B.\u00a0Object-Oriented Software Construction, second edition. Prentice Hall\u00a0(1997)."},{"key":"e_1_3_1_30_2","doi-asserted-by":"crossref","unstructured":"Meyer B. Ciupa I. Leitner A. and Liu L.L.\u00a0Automatic testing of object-oriented software. In\u00a0SOFSEM 2007: Theory and Practice of Computer Science: 33rd Conf. on Current Trends in Theory and Practice of Computer Science. Springer\u00a0(2007) 114\u2013129.","DOI":"10.1007\/978-3-540-69507-3_9"},{"key":"e_1_3_1_31_2","unstructured":"Meyer B. et al.\u00a0Systematic evaluation of test failure results. In\u00a0Workshop on Reliability Analysis of System Failure Data\u00a0(2007)."},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3105906"},{"key":"e_1_3_1_33_2","doi-asserted-by":"crossref","unstructured":"M\u00fcller P. and Ruskiewicz J.N.\u00a0Using debuggers to understand failed verification attempts. In\u00a0Intern. Symp. on Formal Methods. Springer\u00a0(2011) 73\u201387.","DOI":"10.1007\/978-3-642-21437-0_8"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF01946816"},{"key":"e_1_3_1_35_2","doi-asserted-by":"crossref","unstructured":"Nilizadeh A. Calvo M. Leavens G.T. and Cok D.R.\u00a0Generating counterexamples in the form of unit yests from Hoarestyle verification attempts. In\u00a0Intern. Conf. on Formal Methods in Software Engineering (FormaliSE). IEEE\u00a0(2022) 124\u2013128.","DOI":"10.1145\/3524482.3527656"},{"key":"e_1_3_1_36_2","doi-asserted-by":"crossref","unstructured":"Nilizadeh A. Calvo M. Leavens G.T. and Le X.B.D.\u00a0More reliable test suites for dynamic APR by using counterexamples. In\u00a0Intern. Symp. on Software Reliability Engineering. IEEE\u00a0(2021) 208\u2013219.","DOI":"10.1109\/ISSRE52982.2021.00032"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/1095430.1081750"},{"key":"e_1_3_1_38_2","doi-asserted-by":"crossref","unstructured":"Tillmann N. and De Halleux J.\u00a0Pex\u2013White Box test generation for .Net. In\u00a0Intern. Conf. on Tests and Proofs. Springer\u00a0(2008) 134\u2013153.","DOI":"10.1007\/978-3-540-79124-9_10"},{"key":"e_1_3_1_39_2","doi-asserted-by":"crossref","unstructured":"Tschannen J. Furia C.A. Nordio M. and Polikarpova N.\u00a0Autoproof: Auto-active functional verification of object-oriented programs. In\u00a0Intern. Conf. on Tools and Algorithms for the Construction and Analysis of Systems. Springer\u00a0(2015) 566\u2013580.","DOI":"10.1007\/978-3-662-46681-0_53"},{"key":"e_1_3_1_40_2","doi-asserted-by":"crossref","unstructured":"Wei Y. et al. Automated fixing of programs with contracts. In\u00a0Proceedings of the 19th Intern. Symp. on Software Testing and Analysis\u00a0(2010) 61\u201372.","DOI":"10.1145\/1831708.1831716"},{"key":"e_1_3_1_41_2","doi-asserted-by":"crossref","unstructured":"Williams N. Marre B. Mouy P. and Roger M.\u00a0Pathcrawler: Automatic generation of path tests by combining static and dynamic analysis. In\u00a0European Dependable Computing Conf. Springer\u00a0(2005) 281\u2013292.","DOI":"10.1007\/11408901_21"}],"container-title":["Communications of the ACM"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3793869","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T21:19:39Z","timestamp":1788297579000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3793869"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,1]]},"references-count":40,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2026,9,1]]}},"alternative-id":["10.1145\/3793869"],"URL":"https:\/\/doi.org\/10.1145\/3793869","relation":{},"ISSN":["0001-0782","1557-7317"],"issn-type":[{"value":"0001-0782","type":"print"},{"value":"1557-7317","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,1]]},"assertion":[{"value":"2025-07-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}