{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T08:13:01Z","timestamp":1778832781295,"version":"3.51.4"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2026,5,14]],"date-time":"2026-05-14T00:00:00Z","timestamp":1778716800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>Redundancy is widely used as a method for fault and intrusion tolerance. However, if the redundant components lack sufficient diversity, potentially dangerous common mode failures may go undetected. To address this issue, the design diversity approach has been proposed in the literature for decades. In this article, we take an innovative approach to this problem by introducing a broader notion of diversity, which leverages Metamorphic Relations (MRs), i.e., necessary properties that must hold among diverse inputs and diverse outputs. We define two generic categories of MRs that establish data diversity and functional diversity. Furthermore, we elaborate on two corresponding logical architectures, paying particular attention to the necessary conditions for the adjudicator component. Finally, we present an initial evaluation of the proposed architectures, which points out the advantages with respect to their counterparts based on the traditional design diversity method, and discuss future research directions for this novel conceptual approach to redundancy-based fault\/intrusion tolerance.<\/jats:p>","DOI":"10.1145\/3772722","type":"journal-article","created":{"date-parts":[[2025,10,23]],"date-time":"2025-10-23T14:34:43Z","timestamp":1761230083000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Using Metamorphic Relations in Redundancy-based Fault\/Intrusion Tolerance"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0687-6655","authenticated-orcid":false,"given":"Felicita Di","family":"Giandomenico","sequence":"first","affiliation":[{"name":"ISTI-CNR, Pisa, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5165-275X","authenticated-orcid":false,"given":"Giulio","family":"Masetti","sequence":"additional","affiliation":[{"name":"Scuola Secondaria di Primo Grado, Monterchi, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4864-2219","authenticated-orcid":false,"given":"Francesca","family":"Lonetti","sequence":"additional","affiliation":[{"name":"ISTI-CNR, Pisa, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8749-1356","authenticated-orcid":false,"given":"Antonia","family":"Bertolino","sequence":"additional","affiliation":[{"name":"GSSI, L\u2019Aquila, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,5,14]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP52600.2021.00023"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/IPDS.1996.540195"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","unstructured":"Paul Eric Ammann and John C. Knight. 1988. Data diversity: An approach to software fault tolerance. IEEE Transactions on Computers 37 4 (1988) 418\u2013425. DOI: 10.1109\/12.218510.1109\/12.2185","DOI":"10.1109\/12.218510.1109\/12.2185"},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","unstructured":"Algirdas Avizienis. 1985. The N-version approach to fault-tolerant software. IEEE Transactions on Software Engineering 11 12 (1985) 1491\u20131501.","DOI":"10.1109\/TSE.1985.231893"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","unstructured":"Algirdas Avizienis J.-C. Laprie Brian Randell and Carl Landwehr. 2004. Basic concepts and taxonomy of dependable and secure computing. IEEE Transactions on Dependable and Secure Computing 1 1 (2004) 11\u201333.","DOI":"10.1109\/TDSC.2004.2"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/FTCS.1988.5291"},{"key":"e_1_3_2_8_2","first-page":"104","volume-title":"Proceedings of the IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","author":"Ayerdi Jon","unstructured":"Jon Ayerdi, Sergio Segura, Aitor Arrieta, Goiuria Sagardui, and Maite Arratibel. 2020. QoS-aware metamorphic testing: An elevation case study. In Proceedings of the IEEE 31st International Symposium on Software Reliability Engineering (ISSRE), 104\u2013114."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3473920"},{"key":"e_1_3_2_10_2","doi-asserted-by":"crossref","unstructured":"Jon Ayerdi Valerio Terragni Gunel Jahangirova Aitor Arrieta and Paolo Tonella. 2024. GenMorph: Automatically generating metamorphic relations via genetic programming. IEEE Transactions on Software Engineering 50 7 (2024) 1888\u20131900.","DOI":"10.1109\/TSE.2024.3407840"},{"key":"e_1_3_2_11_2","doi-asserted-by":"crossref","unstructured":"Benoit Baudry and Martin Monperrus. 2015. The multiple facets of software diversity: Recent developments in year 2000 and beyond. ACM Computing Surveys 48 1 (2015) 1\u201326.","DOI":"10.1145\/2807593"},{"key":"e_1_3_2_12_2","unstructured":"Andrea Bondavalli Felicita Di Giandomenico and Jie Xu. 1993. A cost-effective and flexible scheme for software fault tolerance. Journal of Computer Systems Science and Engineering 8 (1993) 234\u2013244."},{"key":"e_1_3_2_13_2","unstructured":"Nazanin Bayati Chaleshtari Fabrizio Pastore and Yoann Marquer. 2025. Catalog of Metamorphic Relations for MST. Retrieved from https:\/\/github.com\/MetamorphicSecurityTesting\/CWE"},{"key":"e_1_3_2_14_2","unstructured":"Tsong Yueh Chen Shing C. Cheung and Shiu Ming Yiu. 2020. Metamorphic testing: A new approach for generating next test cases. arXiv:2002.12543. Retrieved from https:\/\/arxiv.org\/abs\/2002.12543"},{"key":"e_1_3_2_15_2","doi-asserted-by":"crossref","unstructured":"Tsong Yueh Chen Fei-Ching Kuo Huai Liu Pak-Lok Poon Dave Towey T. H. Tse and Zhi Quan Zhou. 2018. Metamorphic testing: A review of challenges and opportunities. ACM Compututing Surveys 51 1 (2018) 1\u201327.","DOI":"10.1145\/3143561"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/QRS62785.2024.00020"},{"key":"e_1_3_2_17_2","unstructured":"Guowei Dong Baowen Xu Lin Chen Changhai Nie and Lulu Wang. 2008. Case studies on testing with compositional metamorphic relations. Journal of Southeast University (English Edition) 24 4 (2008) 437\u2013443."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213858"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/EDCC.2016.27"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2013.6698899"},{"key":"e_1_3_2_21_2","doi-asserted-by":"crossref","unstructured":"Upulee Kanewala James M. Bieman and Asa Ben-Hur. 2016. Predicting metamorphic relations for testing scientific software: a machine learning approach using graph kernels. Software Testing Verification and Reliability 26 3 (2016) 245\u2013269.","DOI":"10.1002\/stvr.1594"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.5555\/2184121.2184147"},{"key":"e_1_3_2_23_2","unstructured":"J. R. Lamarsh. 1966. Introduction to Nuclear Reactor Theory. Addison-Wesley Reading MA."},{"key":"e_1_3_2_24_2","unstructured":"J. R. Lamarsh and A. J. Baratta. 2001. Introduction to Nuclear Engineering (3rd ed.). Prentice-Hall Upper Saddle River NJ."},{"key":"e_1_3_2_25_2","doi-asserted-by":"crossref","unstructured":"Jean-Claude Laprie Jean Arlat Christian Beounes and Karama Kanoun. 1990. Definition and analysis of hardware- and software-fault-tolerant architectures. Computer 23 7 (1990) 39\u201351.","DOI":"10.1109\/2.56851"},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","unstructured":"Gholamreza Latif-Shabgahi Julian M Bass and Stuart Bennett. 2004. A taxonomy for software voting algorithms used in safety-critical systems. IEEE Transactions on Reliability 53 3 (2004) 319\u2013328.","DOI":"10.1109\/TR.2004.832819"},{"key":"e_1_3_2_27_2","doi-asserted-by":"crossref","unstructured":"Rui Li Huai Liu Pak-Lok Poon Dave Towey Chang-Ai Sun Zheng Zheng Zhi Quan Zhou and Tsong Yueh Chen. 2025. Metamorphic relation generation: State of the art and research directions. ACM Transactions on Software Engineering and Methodology 34 5 (2025) 1\u201325.","DOI":"10.1145\/3708521"},{"key":"e_1_3_2_28_2","doi-asserted-by":"crossref","unstructured":"Bev Littlewood and Lorenzo Strigini. 2000. A Discussion of Practices for Enhancing Diversity in Software Designs. Technical Report DISPO. Centre for Software Reliability City University.","DOI":"10.1145\/336512.336551"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30108-0_26"},{"key":"e_1_3_2_30_2","doi-asserted-by":"crossref","unstructured":"Huai Liu Fei-Ching Kuo Dave Towey and Tsong Yueh Chen. 2013. How effectively does metamorphic testing alleviate the oracle problem? IEEE Transactions on Software Engineering 40 1 (2013) 4\u201322.","DOI":"10.1109\/TSE.2013.46"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2591062.2591109"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN.2008.4630087"},{"key":"e_1_3_2_33_2","doi-asserted-by":"crossref","unstructured":"Brian Randell. 1975. System structure for software fault tolerance. IEEE Transactions on Software Engineering 1 2 (1975) 220\u2013232.","DOI":"10.1109\/TSE.1975.6312842"},{"key":"e_1_3_2_34_2","unstructured":"Brian Randell and Jie Xu. 1994. The Evolution of the Recovery Block Concept. Vol. 3. John Wiley & Sons Ltd. Chichester 1\u201322."},{"key":"e_1_3_2_35_2","unstructured":"Walter Rudin. 1976. Principles of Mathematical Analysis Vol. 3."},{"key":"e_1_3_2_36_2","doi-asserted-by":"crossref","unstructured":"Kizito Salako and Lorenzo Strigini. 2013. When does \u201cDiversity\u201d\u2019 in development reduce common failures? Insights from probabilistic modeling. IEEE Transactions on dependable and secure computing 11 2 (2013) 193\u2013206.","DOI":"10.1109\/TDSC.2013.32"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524846.3527338"},{"key":"e_1_3_2_38_2","doi-asserted-by":"crossref","unstructured":"Sergio Segura Gordon Fraser Ana B. Sanchez and Antonio Ruiz-Cort\u00e9s. 2016. A survey on metamorphic testing. IEEE Transactions on Software Engineering 42 9 (2016) 805\u2013824.","DOI":"10.1109\/TSE.2016.2532875"},{"key":"e_1_3_2_39_2","doi-asserted-by":"crossref","unstructured":"Sergio Segura Dave Towey Zhi Quan Zhou and Tsong Yueh Chen. 2018. Metamorphic testing: Testing the untestable. IEEE Software 37 3 (2018) 46\u201353.","DOI":"10.1109\/MS.2018.2875968"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-70245-7_9"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/RELDIS.1991.145409"},{"key":"e_1_3_2_42_2","unstructured":"Chang-Ai Sun An Fu Pak-Lok Poon Xiaoyuan Xie Huai Liu and Tsong Yueh Chen. 2019. METRIC \\({}^{+}\\) : A metamorphic relation identification technique based on input plus output domains. IEEE Transactions on Software Engineering 47 9 (2019) 1764\u20131785."},{"key":"e_1_3_2_43_2","doi-asserted-by":"crossref","unstructured":"Pascal Traverse Isabelle Lacaze and Jean Souyris. 2004. Airbus fly-by-wire: A total approach to dependability. In Building the Information Society. Ren\u00e8 Jacquart (Ed.) Springer Boston MA 191\u2013212.","DOI":"10.1007\/978-1-4020-8157-6_18"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-45177-3_1"},{"key":"e_1_3_2_45_2","unstructured":"J. Von Neumann. 1956. Probabilistic Logics Vol. 34. Priceton University Press."},{"key":"e_1_3_2_46_2","unstructured":"Richard Thomas Wood Randy Belles Mustafa Sacit Cetiner David Eugene Holcomb Kofi Korsah Andy Loebl Gary T. Mays Michael David Muhlheim James Allen Mullens Willis P. Poore III et al. 2010. Diversity Strategies for Nuclear Power Plant Instrumentation and Control Systems. Technical Report. Oak Ridge National Lab.(ORNL) Oak Ridge TN."},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/AERO.1996.495891"},{"key":"e_1_3_2_48_2","first-page":"235","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME)","author":"Zhang Bo","year":"2019","unstructured":"Bo Zhang, Hongyu Zhang, Junjie Chen, Dan Hao, and Pablo Moscato. 2019. Automatic discovery and cleansing of numerical metamorphic relations. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME), 235\u2013245."},{"key":"e_1_3_2_49_2","doi-asserted-by":"crossref","unstructured":"Zheng Zheng Daixu Ren Huai Liu and Tsong Yueh Chen. 2024. Metamorphic fault tolerance: Addressing the oracle problem of reliability assurance for contemporary software systems. Computer 57 7 (2024) 77\u201386.","DOI":"10.1109\/MC.2024.3390759"},{"key":"e_1_3_2_50_2","doi-asserted-by":"crossref","unstructured":"Zhi Quan Zhou ShuJia Zhang Markus Hagenbuchner T. H. Tse Fei-Ching Kuo and Tsong Yueh Chen. 2012. Automated functional testing of online search services. Software Testing Verification and Reliability 22 4 (2012) 221\u2013243.","DOI":"10.1002\/stvr.437"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3772722","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T07:49:29Z","timestamp":1778831369000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3772722"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,14]]},"references-count":49,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3772722"],"URL":"https:\/\/doi.org\/10.1145\/3772722","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,14]]},"assertion":[{"value":"2025-02-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-11","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-14","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}