{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T03:47:03Z","timestamp":1760586423756,"version":"3.41.0"},"reference-count":24,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2006,11,1]],"date-time":"2006-11-01T00:00:00Z","timestamp":1162339200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGSOFT Softw. Eng. Notes"],"published-print":{"date-parts":[[2006,11]]},"abstract":"<jats:p>User configurable software systems allow users to customize functionality at run time. In essence, each such system consists of a family of potentially thousands or millions of program instantiations. Testing methods cannot test all of these configurations, therefore some sampling mechanism must be applied. A common approach to providing such a mechanism has been to use combinatorial interaction testing. To date, however, little work has been done to quantify the effects of different configurations on a test suites' operation and effectiveness. In this paper we present a case study that investigates the effects of changing configurations on two types of test suites. Our results show that test coverage and fault detection effectiveness do not vary much across configurations for entire test suites; however, for individual test cases and certain types of faults, configurations matter.<\/jats:p>","DOI":"10.1145\/1218776.1218785","type":"journal-article","created":{"date-parts":[[2007,4,5]],"date-time":"2007-04-05T19:52:18Z","timestamp":1175802738000},"page":"1-9","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":62,"title":["Testing across configurations"],"prefix":"10.1145","volume":"31","author":[{"given":"Myra B.","family":"Cohen","sequence":"first","affiliation":[{"name":"University of Nebraska-Lincoln, Lincoln, NE"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joshua","family":"Snyder","sequence":"additional","affiliation":[{"name":"University of Nebraska-Lincoln, Lincoln, NE"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gregg","family":"Rothermel","sequence":"additional","affiliation":[{"name":"University of Nebraska-Lincoln, Lincoln, NE"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2006,11]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1992.tb00164.x"},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the International Conference on Software Testing Analysis & Review","author":"Burr K.","year":"1998","unstructured":"K. Burr and W. Young . Combinatorial test techniques: Table-based automation, test generation and code coverage . In Proceedings of the International Conference on Software Testing Analysis & Review , 1998 . K. Burr and W. Young. Combinatorial test techniques: Table-based automation, test generation and code coverage. In Proceedings of the International Conference on Software Testing Analysis & Review, 1998."},{"key":"e_1_2_1_3_1","first-page":"745","volume-title":"IEEE International Conference on Communications","author":"Burroughs K.","year":"1994","unstructured":"K. Burroughs , A. Jain , and R. L. Erickson . Improved quality of protocol testing through techniques of experimental design. In Supercomm\/IC , IEEE International Conference on Communications , pages 745 -- 752 , 1994 . K. Burroughs, A. Jain, and R. L. Erickson. Improved quality of protocol testing through techniques of experimental design. In Supercomm\/IC, IEEE International Conference on Communications, pages 745--752, 1994."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.605761"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/52.536462"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5555\/851015.856168"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/302405.302640"},{"key":"e_1_2_1_8_1","first-page":"84","volume-title":"Proceedings of the IEEE Workshop on Industrial Strength Formal Specification Techniques","author":"Dalal S. R.","year":"1998","unstructured":"S. R. Dalal , A. Jain , G. Patton , M. Rathi , and P. Seymour . AETGSM web: a Web based service for automatic efficient test generation from functional requirements . In Proceedings of the IEEE Workshop on Industrial Strength Formal Specification Techniques , pages 84 -- 85 , Oct 1998 . S. R. Dalal, A. Jain, G. Patton, M. Rathi, and P. Seymour. AETGSM web: a Web based service for automatic efficient test generation from functional requirements. In Proceedings of the IEEE Workshop on Industrial Strength Formal Specification Techniques, pages 84--85, Oct 1998."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/253228.253271"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2001.972727"},{"key":"e_1_2_1_11_1","first-page":"1","volume-title":"USENIX Windows System Symposium","author":"Forrester J. E.","year":"2000","unstructured":"J. E. Forrester and B. P. Miller . An empirical study of the robustness of Windows NT applications using random testing . In USENIX Windows System Symposium , pages 1 -- 10 , July 2000 . J. E. Forrester and B. P. Miller. An empirical study of the robustness of Windows NT applications using random testing. In USENIX Windows System Symposium, pages 1--10, July 2000."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.319"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1049\/ic:20040349"},{"key":"e_1_2_1_14_1","first-page":"91","volume-title":"Proceedings of the NASA\/IEEE Software Engineering Workshop","author":"Kuhn D.","year":"2002","unstructured":"D. Kuhn and M. Reilly . An investigation of the applicability of design of experiments to software testing . In Proceedings of the NASA\/IEEE Software Engineering Workshop , pages 91 -- 95 , 2002 . D. Kuhn and M. Reilly. An investigation of the applicability of design of experiments to software testing. In Proceedings of the NASA\/IEEE Software Engineering Workshop, pages 91--95, 2002."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2004.24"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/4372.4375"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.5555\/998675.999451"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1145735.1145743"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/62959.62964"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.536955"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/52.714817"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1007512.1007519"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2006.8"},{"key":"e_1_2_1_24_1","unstructured":"C. You. My IE3.1. http:\/\/www.myie2.com\/html_en\/home.htm.  C. You. My IE3.1. http:\/\/www.myie2.com\/html_en\/home.htm."}],"container-title":["ACM SIGSOFT Software Engineering Notes"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1218776.1218785","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1218776.1218785","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T20:22:41Z","timestamp":1750278161000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1218776.1218785"}},"subtitle":["implications for combinatorial testing"],"short-title":[],"issued":{"date-parts":[[2006,11]]},"references-count":24,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2006,11]]}},"alternative-id":["10.1145\/1218776.1218785"],"URL":"https:\/\/doi.org\/10.1145\/1218776.1218785","relation":{},"ISSN":["0163-5948"],"issn-type":[{"type":"print","value":"0163-5948"}],"subject":[],"published":{"date-parts":[[2006,11]]},"assertion":[{"value":"2006-11-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}