{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T13:00:11Z","timestamp":1788872411676,"version":"build-2803163510"},"reference-count":83,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2011,9,1]],"date-time":"2011-09-01T00:00:00Z","timestamp":1314835200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100002855","name":"Ministry of Science and Technology of the People's Republic of China","doi-asserted-by":"publisher","award":["2008AAO1Z1432009AAO1Z147"],"award-info":[{"award-number":["2008AAO1Z1432009AAO1Z147"]}],"id":[{"id":"10.13039\/501100002855","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["6.08E+23"],"award-info":[{"award-number":["6.08E+23"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2011,9]]},"abstract":"<jats:p>Combinatorial Testing (CT) involves the design of a small test suite to cover the parameter value combinations so as to detect failures triggered by the interactions among these parameters. To make full use of CT and to extend its advantages, this article first gives a model of CT and then presents a theory of the Minimal Failure-causing Schema (MFS), including the concept of the MFS, proof of its existence, some of its properties, and a method of finding the MFS. Then we propose a methodology for CT based on this MFS theory and the existing research. Our MFS-based methodology emphasizes that CT should work on accurate testing requirements, and has the following advantages: 1) Detect failure to the greatest degree with the least cost. 2) Effectiveness is improved by emphasizing mining of the information in software and making full use of the information gained from test design and execution. 3) Determine the root causes of failures and reveal related faults near the exposed ones. 4) Provide a foundation and model for regression testing and software quality evaluation of CT. A case study is presented to illustrate the MFS-based CT methodology, and an empirical study on a real software developed by us shows that the MFS really exists and the methodology based on MFS can considerably improve CT.<\/jats:p>","DOI":"10.1145\/2000799.2000801","type":"journal-article","created":{"date-parts":[[2011,9,27]],"date-time":"2011-09-27T10:02:19Z","timestamp":1317117739000},"page":"1-38","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":44,"title":["The Minimal Failure-Causing Schema of Combinatorial Testing"],"prefix":"10.1145","volume":"20","author":[{"given":"Changhai","family":"Nie","sequence":"first","affiliation":[{"name":"Nanjing University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hareton","family":"Leung","sequence":"additional","affiliation":[{"name":"Hong Kong Polytechnic University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2011,9]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the 22nd Pacific Northwest Software Quality Conference. 180--196","author":"Bach J."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2003.1232283"},{"key":"e_1_2_1_3_1","first-page":"100","article-title":"Pairwise testing","volume":"78","author":"Blass A.","year":"2002","journal-title":"Bull. Eur. Assn. Theor. Comput. Sci."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1992.tb00164.x"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1083274.1083275"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1062455.1062495"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the International Conference on Software Testing Analysis and Review. 503--513","author":"Burr K."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.5555\/1792786.1792794"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the 3rd Workshop on Automated Formal Methods. 1--10","author":"Calvagna A."},{"key":"e_1_2_1_10_1","volume-title":"Proceedings of the 3rd International Conference on Quality Software (QSIC\u201903)","author":"Cheng C."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1520-6610(1998)6:6<411::AID-JCD3>3.0.CO;2-I"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the 5th IEEE International Symposium on Software Reliability Engineering. IEEE Press, 303--309","author":"Cohen D. M."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/52.536462"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.605761"},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","unstructured":"Cohen M. B. 2004. Designing test suites for software interaction testing. Ph.D. thesis University of Auckland New Zealand. Cohen M. B. 2004. Designing test suites for software interaction testing. Ph.D. thesis University of Auckland New Zealand.","DOI":"10.1109\/ICSE.2003.1201186"},{"key":"e_1_2_1_16_1","volume-title":"Proceedings of the 14th International Symposium on Software Reliability Engineering (ISSRE\u201903)","author":"Cohen M. B."},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 25th International Conference on Software Engineering (ICSE\u201903)","author":"Cohen M. B."},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the 27th Annual International. Computer Software and Applications Conference (COMPSAC\u201903)","author":"Cohen M."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1218776.1218785"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques (TAICPART-MUTATION\u201907)","author":"Cohen M. B."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273463.1273482"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2008.50"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10878-007-9082-4"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the 24th Pacific Northwest Software Quality Conference.","author":"Czerwonka J.","year":"2006"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the 9th International Symposium on Software Reliability Engineering (ISSRE\u201998)","author":"Dalal S."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/302405.302640"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.2307\/1271179"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1002\/1520-6610(2000)8:4<239::AID-JCD3>3.0.CO;2-C"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-24587-2_32"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1287624.1287709"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the Congress on Evolutionary Computation (CEC\u201903)","author":"Ghazi S. A."},{"key":"e_1_2_1_32_1","volume-title":"Proceedings of the 25th Conference on IASTED International Multi-Conference (SE\u201907)","author":"Grindal M."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.319"},{"key":"e_1_2_1_34_1","unstructured":"Grindal M. Offutt J. and Mellin J. 2006a. Handling constraints in the input space when using combination strategies for software testing. Tech. rep. HS-IKI-TR-06-001. University of Sk\u00f6vde. Grindal M. Offutt J. and Mellin J. 2006a. Handling constraints in the input space when using combination strategies for software testing. Tech. rep. HS-IKI-TR-06-001. University of Sk\u00f6vde."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-006-9024-2"},{"key":"e_1_2_1_36_1","first-page":"237","article-title":"Software and hardware testing using combinatorial covering suitess","volume":"6","author":"Hartman A.","year":"2002","journal-title":"Graph Theor. Comb. Algor."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.disc.2003.11.029"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/11402763_13"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10601-006-7094-9"},{"key":"e_1_2_1_40_1","volume-title":"Proceedings of the International Council on Systems Engineering Conference.","author":"Huller J.","year":"2000"},{"key":"e_1_2_1_41_1","volume-title":"Proceedings of the Asia and South Pacific Design Automation Conference.","author":"Kahng A."},{"key":"e_1_2_1_42_1","unstructured":"Kobayashi N. 2002. Design and evaluation of automatic test generation strategies for functional testing of software. Ph.D. thesis Osaka University Osaka Japan. Kobayashi N. 2002. Design and evaluation of automatic test generation strategies for functional testing of software. Ph.D. thesis Osaka University Osaka Japan."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-0190(01)00195-8"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/1241572.1241582"},{"key":"e_1_2_1_45_1","volume-title":"Proceedings of the 27th Annual NASA Goddard Software Engineering Workshop (SEW\u201902)","author":"Kuhn D. R."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2004.24"},{"key":"e_1_2_1_47_1","unstructured":"Lei Y. and Tai K. C. 2001. In-parameter-oder: a test generation strategy for pairwise testing. Tech. rep. TR-2001-03 5 109--111. Lei Y. and Tai K. C. 2001. In-parameter-oder: a test generation strategy for pairwise testing. Tech. rep. TR-2001-03 5 109--111."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/ECBS.2007.47"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/1082983.1083281"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2005.23"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/4372.4375"},{"key":"e_1_2_1_52_1","unstructured":"Myers G. J. Sandler C. Badgett T. and Thomas T. M. 2004. The Art of Software Testing 2nd Ed. John Wiley and Sons 3--10. Myers G. J. Sandler C. Badgett T. and Thomas T. M. 2004. The Art of Software Testing 2nd Ed. John Wiley and Sons 3--10."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/11558569_15"},{"key":"e_1_2_1_54_1","volume-title":"Proceedings of the International Conference on Software Engineering and Knowledge Engineering.","author":"Nie C."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/QSIC.2006.31"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0166-218X(03)00291-9"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/62959.62964"},{"key":"e_1_2_1_58_1","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance (ICSM). IEEE, 413--418","author":"Qu X."},{"key":"e_1_2_1_59_1","unstructured":"Salem A. M. 2001. A software testing model: using design of experiments (doe) and logistic regression. Ph.D. thesis Florida Institute of Technology Melbourne FL. Salem A. M. 2001. A software testing model: using design of experiments (doe) and logistic regression. Ph.D. thesis Florida Institute of Technology Melbourne FL."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2003.10.008"},{"key":"e_1_2_1_61_1","unstructured":"Schroeder P. J. 2002. Black-box test reduction using input-output analysis. Ph.D. thesis Illinois Institute of Technology Chicago IL. Schroeder P. J. 2002. Black-box test reduction using input-output analysis. Ph.D. thesis Illinois Institute of Technology Chicago IL."},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/347636.349042"},{"key":"e_1_2_1_63_1","volume-title":"Proceedings of the Automated Software Engineering Doctoral Symposium.","author":"Schroeder P. J."},{"key":"e_1_2_1_64_1","article-title":"Generating expected results for automated black-box testing","author":"Schroeder P. J.","year":"2002","journal-title":"IEEE Trans. Autom. Sci. Engin. 139--148."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.5555\/1018434.1021565"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/18.6031"},{"key":"e_1_2_1_67_1","first-page":"202","article-title":"Covering arrays of higher strength from permutation vectors","volume":"3","author":"Sherwood G. B.","year":"2005","journal-title":"J. Comb. Des."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1007\/11428862_179"},{"key":"e_1_2_1_69_1","volume-title":"Proceedings of the 28th Annual International Computer Software and Applications Conference (COMPSAC\u201904)","author":"Shiba T."},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.979992"},{"key":"e_1_2_1_71_1","volume-title":"Proceedings of the IEEE Aerospace Conference. 431--437","author":"Tung Y.-W."},{"key":"e_1_2_1_72_1","unstructured":"Turban R. C. 2006. Algorithms for covering arrays. Ph.D. thesis Tempe Arizona State University. Turban R. C. 2006. Algorithms for covering arrays. Ph.D. thesis Tempe Arizona State University."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/1295074.1295085"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1109\/QSIC.2008.52"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.5555\/851024.856239"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.5555\/648131.748027"},{"key":"e_1_2_1_77_1","unstructured":"Williams A. W. 2002. Software component interaction testing: coverage measurement and generation of configurations. Ph.D. thesis University of Ottawa Canada. Williams A. W. 2002. Software component interaction testing: coverage measurement and generation of configurations. Ph.D. thesis University of Ottawa Canada."},{"key":"e_1_2_1_78_1","volume-title":"Proceedings of the 7th International Symposium on Software Reliability Engineering (ISSRE\u201996)","author":"Williams A. W."},{"key":"e_1_2_1_79_1","volume-title":"Proceedings of the ACS\/IEEE International Conference on Computer Systems and Applications (AICCSA\u201901)","author":"Williams A. W."},{"key":"e_1_2_1_80_1","volume-title":"Proceedings of the IFIP 14th International Conference on Testing Communicating Systems (TestCom\u201902)","author":"Williams A. W."},{"key":"e_1_2_1_81_1","unstructured":"Xu B. Nie C. Shi L. Chu W. C. Yang H. and Chen H. 2003a. Test plan design for software configuration testing. In Software Engineering Research and Practice. CSREA Press 686--692. Xu B. Nie C. Shi L. Chu W. C. Yang H. and Chen H. 2003a. Test plan design for software configuration testing. In Software Engineering Research and Practice . CSREA Press 686--692."},{"key":"e_1_2_1_82_1","volume-title":"Proceedings of the International Conference on Web Engineering (ICWE). Springer, 310--313","author":"Xu L."},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2006.8"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2000799.2000801","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2000799.2000801","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T07:00:03Z","timestamp":1750230003000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2000799.2000801"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,9]]},"references-count":83,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2011,9]]}},"alternative-id":["10.1145\/2000799.2000801"],"URL":"https:\/\/doi.org\/10.1145\/2000799.2000801","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,9]]},"assertion":[{"value":"2009-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2009-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2011-09-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}