{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,6]],"date-time":"2026-11-06T06:09:43Z","timestamp":1793945383257,"version":"4.8.0"},"reference-count":63,"publisher":"Association for Computing Machinery (ACM)","issue":"8","license":[{"start":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T00:00:00Z","timestamp":1784160000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,8,31]]},"abstract":"<jats:p>\n                    N-Version Programming is a well-known methodology for developing fault-tolerant systems. It achieves fault detection and correction at runtime by adding diverse redundancy into programs, minimizing fault mode overlap between redundant program variants. In this work, we propose the automated generation of program variants using large language models. We design, develop, and evaluate\n                    <jats:sc>Gal\u00e1pagos<\/jats:sc>\n                    : a tool for generating program variants using LLMs, validating their correctness and equivalence, and using them to assemble N-Version binaries. We evaluate\n                    <jats:sc>Gal\u00e1pagos<\/jats:sc>\n                    by creating N-Version components of real-world C code. Our original results show that\n                    <jats:sc>Gal\u00e1pagos<\/jats:sc>\n                    can produce program variants that are proven to be functionally equivalent, even when the variants are written in a different programming language. Our systematic diversity measurement indicates that functionally equivalent variants produced by\n                    <jats:sc>Gal\u00e1pagos<\/jats:sc>\n                    are statically different after compilation and present diverging internal behavior at runtime. We demonstrate that the variants produced by\n                    <jats:sc>Gal\u00e1pagos<\/jats:sc>\n                    can protect C code against real miscompilation bugs which affect the Clang compiler. Overall, our article shows that producing N-Version software can be drastically automated by advanced usage of practical formal verification and generative language models.\n                  <\/jats:p>","DOI":"10.1145\/3785363","type":"journal-article","created":{"date-parts":[[2025,12,17]],"date-time":"2025-12-17T15:56:57Z","timestamp":1765987017000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["<scp>gal\u00e1pagos<\/scp>\n                    : Automated N-Version Programming with LLMs"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6988-3102","authenticated-orcid":false,"given":"Javier","family":"Ron","sequence":"first","affiliation":[{"name":"KTH Royal Institute of Technology, Stockholm, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-1338-3295","authenticated-orcid":false,"given":"Diogo","family":"Gaspar","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology, Stockholm, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9399-8647","authenticated-orcid":false,"given":"Javier","family":"Cabrera-Arteaga","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology, Stockholm, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4015-4640","authenticated-orcid":false,"given":"Benoit","family":"Baudry","sequence":"additional","affiliation":[{"name":"Universit\u00e9 de Montr\u00e9al, Montreal, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3505-3383","authenticated-orcid":false,"given":"Martin","family":"Monperrus","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology, Stockholm, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,7,16]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/HOTOS.1997.595185"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1985.231893"},{"key":"e_1_3_2_4_2","first-page":"23","article-title":"The methodology of n-version programming","volume":"3","author":"Avizienis Algirdas","year":"1995","unstructured":"Algirdas Avizienis. 1995. The methodology of n-version programming. Software Fault Tolerance 3 (1995), 23\u201346.","journal-title":"Software Fault Tolerance"},{"issue":"12","key":"e_1_3_2_5_2","first-page":"96","article-title":"An experimental evaluation of the assumption of independence in multiversion programming","author":"Knight John C.","year":"2012","unstructured":"John C. Knight and Nancy G. Leveson. 2012. An experimental evaluation of the assumption of independence in multiversion programming. IEEE Transactions on Software Engineering SE-12 (1986), 96\u2013109.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.44387"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2001.936213"},{"key":"e_1_3_2_8_2","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde de Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman et al. 2021. Evaluating large language models trained on code. arXiv:2107.03374. Retrieved from https:\/\/arxiv.org\/abs\/2107.03374"},{"key":"e_1_3_2_9_2","unstructured":"D\u00e1vid Hidv\u00e9gi Khashayar Etemadi Sofia Bobadilla and Martin Monperrus. 2024. CigaR: Cost-efficient program repair with LLMs. arXiv:2402.06598. Retrieved from https:\/\/arxiv.org\/abs\/2402.06598"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3559555"},{"key":"e_1_3_2_11_2","unstructured":"Christian Collberg. The Tigress C Obfuscator. Retrieved January 5 2025 from https:\/\/tigress.wtf"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2024.103731"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/1993498.1993532"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1985.231895"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2023.3346195"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-08-034801-8.50012-X"},{"key":"e_1_3_2_17_2","article-title":"Proving and rewarding client diversity to strengthen resilience of blockchain networks","author":"Ron Javier","year":"2025","unstructured":"Javier Ron, Zheyuan He, and Martin Monperrus. 2025. Proving and rewarding client diversity to strengthen resilience of blockchain networks. ACM Distributed Ledger Technologies: Research and Practice (2025).","journal-title":"ACM Distributed Ledger Technologies: Research and Practice"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3358960.3379125"},{"key":"e_1_3_2_19_2","first-page":"271","volume-title":"Proceedings of the 2017 USENIX Annual Technical Conference (USENIX ATC \u201917)","author":"Xu Meng","year":"2017","unstructured":"Meng Xu, Kangjie Lu, Taesoo Kim, and Wenke Lee. 2017. Bunshin: Compositing security mechanisms through diversification. In Proceedings of the 2017 USENIX Annual Technical Conference (USENIX ATC \u201917), 271\u2013283."},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN53405.2022.00049"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ETFA.2017.8247678"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/2786763.2694390"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/2899015.2899026"},{"key":"e_1_3_2_24_2","unstructured":"Elijah Pelofske Vincent Urias and Lorie M. Liebrock. 2024. Automated creation of source code variants of a cryptographic hash function implementation using generative pre-trained transformer models. arXiv:2404.15681. Retrieved from https:\/\/arxiv.org\/abs\/2404.15681"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00021"},{"issue":"9","key":"e_1_3_2_26_2","first-page":"1943","article-title":"SequenceR: Sequence-to-sequence learning for end-to-end program repair","volume":"47","author":"Chen Zimin","year":"2021","unstructured":"Zimin Chen, Steve Kommrusch, Michele Tufano, Louis-No\u00ebl Pouchet, Denys Poshyvanyk, and Martin Monperrus. 2021. SequenceR: Sequence-to-sequence learning for end-to-end program repair. IEEE Transactions on Software Engineering 47, 9 (2021), 1943\u20131959.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_27_2","unstructured":"Marie-Anne Lachaux Baptiste Roziere Lowik Chanussot and Guillaume Lample. 2020. Unsupervised translation of programming languages. arXiv:2006.03511. Retrieved from https:\/\/arxiv.org\/abs\/2006.03511"},{"key":"e_1_3_2_28_2","unstructured":"Aidan Z. H. Yang Yoshiki Takashima Brandon Paulsen Josiah Dodds and Daniel Kroening. 2024. VERT: Verified equivalent rust transpilation with large language models as few-shot learners. arXiv:2404.18852. Retrieved from https:\/\/arxiv.org\/abs\/2404.18852"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931074"},{"key":"e_1_3_2_30_2","volume-title":"The CompCert C Verified Compiler: Documentation and User\u2019s Manual","year":"2023","unstructured":"Xavier Leroy. 2023. The CompCert C Verified Compiler: Documentation and User\u2019s Manual. Technical report, Inria."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3360581"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/1450058.1450093"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650105.3652302"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/CGO.2013.6494997"},{"key":"e_1_3_2_35_2","first-page":"39","volume-title":"Proceedings of the 21st USENIX Conference on Security Symposium (USENIX Security \u201912)","author":"Kemerlis Vasileios P.","year":"2012","unstructured":"Vasileios P. Kemerlis, Georgios Portokalidis, and Angelos D. Keromytis. 2012. kGuard: Lightweight kernel protection against return-to-user attacks. In Proceedings of the 21st USENIX Conference on Security Symposium (USENIX Security \u201912). USENIX Association, 39."},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.52202\/075280-0943"},{"key":"e_1_3_2_37_2","unstructured":"Fang Liu Yang Liu Lin Shi Houkun Huang Ruifeng Wang Zhen Yang and Li Zhang. 2024. Exploring and evaluating hallucinations in LLM-powered code generation. arXiv:2404.00971. Retrieved from https:\/\/arxiv.org\/abs\/2404.00971"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3453483.3454030"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510457.3513031"},{"key":"e_1_3_2_40_2","first-page":"1335","volume-title":"Proceedings of the 27th USENIX Security Symposium (USENIX Security \u201918)","author":"Breidenbach Lorenz","year":"2018","unstructured":"Lorenz Breidenbach, Phil Daian, Florian Tram\u00e8r, and Ari Juels. 2018. Enter the hydra: Towards principled bug bounties and exploit-resistant smart contracts. In Proceedings of the 27th USENIX Security Symposium (USENIX Security \u201918). USENIX Association, 1335\u20131352."},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/2384616.2384680"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/SCAM.2015.7335406"},{"key":"e_1_3_2_43_2","unstructured":"Javier Cabrera Arteaga. 2024. Software Diversification for WebAssembly. Ph.D. thesis. KTH Royal Institute of Technology."},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/1065010.1065034"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1038\/234034a0"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/2420950.2420997"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3547350"},{"key":"e_1_3_2_48_2","volume-title":"Proceedings of the 6th Workshop on Hot Topics in System Dependability","author":"Singh Atul","year":"2010","unstructured":"Atul Singh, Nishant Sinha, and Nitin Agrawal. 2010. Avatars for pennies: Cheap N-version programming for replication. In Proceedings of the 6th Workshop on Hot Topics in System Dependability."},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639226"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2024.3469388"},{"key":"e_1_3_2_51_2","doi-asserted-by":"crossref","unstructured":"Mingzhe Du Anh Tuan Luu Bin Ji Qian Liu and See-Kiong Ng. 2024. Mercury: A code efficiency benchmark for code large language models. Advances in Neural Information Processing Systems 37 (2024) 16601\u201316622.","DOI":"10.52202\/079017-0529"},{"key":"e_1_3_2_52_2","unstructured":"Zibin Zheng Kaiwen Ning Yanlin Wang Jingwen Zhang Dewu Zheng Mingxi Ye and Jiachi Chen. 2024. A survey of large language models for code: Evolution benchmarking and future trends. arXiv:2311.10372. Retrieved from https:\/\/arxiv.org\/abs\/2311.10372"},{"key":"e_1_3_2_53_2","unstructured":"Juyong Jiang Fan Wang Jiasi Shen Sungju Kim and Sunghun Kim. 2024. A survey on large language models for code generation. arXiv:2406.00515. Retrieved from https:\/\/arxiv.org\/abs\/2406.00515"},{"key":"e_1_3_2_54_2","doi-asserted-by":"crossref","unstructured":"Sahil Bhatia Jie Qiu Niranjan Hasabnis Sanjit A. Seshia and Alvin Cheung. 2024. Verified code transpilation with LLMs. Advances in Neural Information Processing Systems 37 (2024) 41394\u201341424.","DOI":"10.52202\/079017-1310"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3660778"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180247"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106269"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635898"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-09940-8_20"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.67"},{"key":"e_1_3_2_61_2","first-page":"19","volume-title":"Proceedings of the 11th Annual Symposium on Information Assurance. NYS Cyber Security Conference","author":"Persaud Bheesham","year":"2016","unstructured":"Bheesham Persaud, Borke Obada-Obieh, Nilofar Mansourzadeh, Ashley Moni, and Anil Somayaji. 2016. FrankenSSL: Recombining cryptographic libraries for software diversity. In Proceedings of the 11th Annual Symposium on Information Assurance. NYS Cyber Security Conference, 19\u201325."},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/1830483.1830653"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/2858965.2814319"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3397822"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3785363","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T15:34:38Z","timestamp":1784216078000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3785363"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,16]]},"references-count":63,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2026,8,31]]}},"alternative-id":["10.1145\/3785363"],"URL":"https:\/\/doi.org\/10.1145\/3785363","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,16]]},"assertion":[{"value":"2025-05-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-12-04","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-07-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}