{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T09:36:50Z","timestamp":1772271410884,"version":"3.50.1"},"reference-count":35,"publisher":"PeerJ","license":[{"start":{"date-parts":[[2022,2,11]],"date-time":"2022-02-11T00:00:00Z","timestamp":1644537600000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:p>Combinatorial interaction testing, which is a technique to verify a system with numerous input parameters, employs a mathematical object called a covering array as a test input. This technique generates a limited number of test cases while guaranteeing a given combinatorial coverage. Although this area has been studied extensively, handling constraints among input parameters remains a major challenge, which may significantly increase the cost to generate covering arrays. In this work, we propose a mathematical operation, called \u201cweaken-product based combinatorial join\u201d, which constructs a new covering array from two existing covering arrays. The operation reuses existing covering arrays to save computational resource by increasing parallelism during generation without losing combinatorial coverage of the original arrays. Our proposed method significantly reduce the covering array generation time by 13\u201396% depending on use case scenarios.<\/jats:p>","DOI":"10.7717\/peerj-cs.720","type":"journal-article","created":{"date-parts":[[2022,2,11]],"date-time":"2022-02-11T04:58:27Z","timestamp":1644555507000},"page":"e720","source":"Crossref","is-referenced-by-count":1,"title":["Accelerating covering array generation by combinatorial join for industry scale software testing"],"prefix":"10.7717","volume":"8","author":[{"given":"Hiroshi","family":"Ukai","sequence":"first","affiliation":[{"name":"Waseda University, Tokyo, Japan"},{"name":"Rakuten Group, Inc., Tokyo, Japan"}]},{"given":"Xiao","family":"Qu","sequence":"additional","affiliation":[{"name":"Independent Researcher, Raleigh, NC, United States of America"}]},{"given":"Hironori","family":"Washizaki","sequence":"additional","affiliation":[{"name":"Waseda University, Tokyo, Japan"}]},{"given":"Yoshiaki","family":"Fukazawa","sequence":"additional","affiliation":[{"name":"Waseda University, Tokyo, Japan"}]}],"member":"4443","published-online":{"date-parts":[[2022,2,11]]},"reference":[{"issue":"8","key":"10.7717\/peerj-cs.720\/ref-1","doi-asserted-by":"publisher","first-page":"1978","DOI":"10.1016\/j.jss.2013.02.061","article-title":"An orchestrated survey of methodologies for automated software test case generation","volume":"86","author":"Anand","year":"2013","journal-title":"Journal of Systems and Software"},{"key":"10.7717\/peerj-cs.720\/ref-2","first-page":"87","article-title":"Construction of variable strength covering array for combinatorial testing using a greedy approach to genetic algorithm","volume":"9","author":"Bansal","year":"2015","journal-title":"e-Informatica"},{"key":"10.7717\/peerj-cs.720\/ref-3","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1109\/ICSTW50294.2020.00036","article-title":"An automata-based generation method for combinatorial sequence testing of finite state machines","volume-title":"2020 IEEE international conference on software testing, verification and validation workshops (ICSTW)","author":"Bombarda","year":"2020"},{"key":"10.7717\/peerj-cs.720\/ref-4","article-title":"Combinatorial interaction testing","author":"Cai","year":"2020"},{"issue":"7","key":"10.7717\/peerj-cs.720\/ref-5","doi-asserted-by":"publisher","first-page":"437","DOI":"10.1109\/32.605761","article-title":"The aetg system: an approach to testing based on combinatorial design","volume":"23","author":"Cohen","year":"1997","journal-title":"IEEE Transactions on Software Engineering"},{"issue":"7","key":"10.7717\/peerj-cs.720\/ref-6","doi-asserted-by":"publisher","first-page":"437","DOI":"10.1109\/32.605761","article-title":"The aetg system: an approach to testing based on combinatorial design","volume":"23","author":"Cohen","year":"1997","journal-title":"IEEE Transactions on Software Engineering"},{"issue":"5","key":"10.7717\/peerj-cs.720\/ref-7","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1109\/TSE.2008.50","article-title":"Constructing interaction test suites for highly-configurable systems in the presence of constraints: a greedy approach","volume":"34","author":"Cohen","year":"2008","journal-title":"IEEE Transactions on Software Engineering"},{"key":"10.7717\/peerj-cs.720\/ref-8","doi-asserted-by":"crossref","DOI":"10.1109\/CMPSAC.2003.1245373","article-title":"A variable strength interaction testing of components","author":"Cohen","year":"2003"},{"key":"10.7717\/peerj-cs.720\/ref-9","article-title":"Advanced combinatorial testing system (ACTS)","author":"Computer Security Research Center (NIST)","year":"2016"},{"key":"10.7717\/peerj-cs.720\/ref-10","article-title":"Pairwise testing in real world","author":"Czerwonka","year":"2006"},{"key":"10.7717\/peerj-cs.720\/ref-11","article-title":"A PICT model example. GitHub","author":"Czerwonka","year":"2015"},{"key":"10.7717\/peerj-cs.720\/ref-12","article-title":"PICT issue 13. GitHub","author":"Czerwonka","year":"2016"},{"key":"10.7717\/peerj-cs.720\/ref-13","article-title":"Software pairwise testing tools","author":"Czerwonka","year":"2018"},{"issue":"1","key":"10.7717\/peerj-cs.720\/ref-14","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1007\/s10664-010-9135-7","article-title":"Evaluating improvements to a meta-heuristic search for constrained interaction testing","volume":"16","author":"Garvin","year":"2011","journal-title":"Empirical Software Engineering"},{"key":"10.7717\/peerj-cs.720\/ref-15","article-title":"Handling constraints in the input space when using combination strategies for software testing","author":"Grindal","year":"2006","journal-title":"Technical report, HSIKI-TR-06-01"},{"key":"10.7717\/peerj-cs.720\/ref-16","first-page":"229","article-title":"Combinatorial methods for modelling composed software systems","author":"Kampel","year":"2017"},{"key":"10.7717\/peerj-cs.720\/ref-17","article-title":"Unable to start windows service after upgrading to 2.248","author":"Kohsuke","year":"2020"},{"key":"10.7717\/peerj-cs.720\/ref-18","first-page":"75","article-title":"Test oracles and test script generation in combinatorial testing","author":"Kruse","year":"2016"},{"key":"10.7717\/peerj-cs.720\/ref-19","volume-title":"Introduction to combinatorial testing","author":"Kuhn","year":"2013","edition":"1"},{"key":"10.7717\/peerj-cs.720\/ref-20","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1109\/COMPSAC.2016.110","article-title":"Estimating t-way fault profile evolution during testing","volume-title":"2016 IEEE 40th annual computer software and applications conference (COMPSAC)","author":"Kuhn","year":"2016"},{"issue":"6","key":"10.7717\/peerj-cs.720\/ref-21","first-page":"22","article-title":"Automated combinatorial test methods \u2013beyond pairwise testing","volume":"21","author":"Kuhn","year":"2008","journal-title":"Journal of Defense Software Engineering"},{"issue":"3","key":"10.7717\/peerj-cs.720\/ref-22","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1002\/stvr.381","article-title":"Ipog-ipog-d: efficient test generation for multi-way combinatorial testing","volume":"18","author":"Lei","year":"2008","journal-title":"Software Testing, Verification and Reliability"},{"key":"10.7717\/peerj-cs.720\/ref-23","article-title":"Fix native library loading in Windows","author":"Netty Project Community","year":"2016"},{"issue":"2","key":"10.7717\/peerj-cs.720\/ref-24","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1145\/1883612.1883618","article-title":"A survey of combinatorial testing","volume":"43","author":"Nie","year":"2011","journal-title":"ACM Computer Survey"},{"key":"10.7717\/peerj-cs.720\/ref-25","article-title":"Sound coming out of GPU very noisy","author":"Nvidia Corporation","year":"2019"},{"key":"10.7717\/peerj-cs.720\/ref-26","first-page":"72","article-title":"Using artificial life techniques to generate test cases for combinatorial testing","author":"Shiba","year":"2004"},{"key":"10.7717\/peerj-cs.720\/ref-27","article-title":"jcunit performance under constraints","author":"Ukai","year":"2017"},{"key":"10.7717\/peerj-cs.720\/ref-28","first-page":"260","article-title":"Reduce test cost by reusing test oracles through combinatorial join","author":"Ukai","year":"2019"},{"key":"10.7717\/peerj-cs.720\/ref-29","first-page":"508","article-title":"Test design as code: JCUnit","author":"Ukai","year":"2017"},{"key":"10.7717\/peerj-cs.720\/ref-30","first-page":"189","article-title":"Test case generation with regular expressions and combinatorial techniques","author":"Usaola","year":"2017"},{"issue":"12","key":"10.7717\/peerj-cs.720\/ref-31","doi-asserted-by":"crossref","first-page":"3173","DOI":"10.4304\/jsw.8.12.3173-3181","article-title":"Generating variable strength covering array for combinatorial software testing with greedy strategy","volume":"8","author":"Wang","year":"2013","journal-title":"JSW"},{"issue":"11","key":"10.7717\/peerj-cs.720\/ref-32","doi-asserted-by":"publisher","first-page":"2549","DOI":"10.1109\/TSE.2019.2955687","article-title":"Comparative analysis of constraint handling techniques for constrained combinatorial testing","volume":"47","author":"Wu","year":"2019","journal-title":"IEEE Transactions on Software Engineering"},{"key":"10.7717\/peerj-cs.720\/ref-33","article-title":"This data corruption bug will shock you","author":"Yoonsik Park","year":"2018"},{"key":"10.7717\/peerj-cs.720\/ref-34","first-page":"242","article-title":"An efficient algorithm for constraint handling in combinatorial test generation","author":"Yu","year":"2013"},{"key":"10.7717\/peerj-cs.720\/ref-35","first-page":"249","article-title":"A composition-based method for combinatorial test design","author":"Zamansky","year":"2017"}],"container-title":["PeerJ Computer Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/peerj.com\/articles\/cs-720.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-720.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-720.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-720.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,2,11]],"date-time":"2022-02-11T04:58:50Z","timestamp":1644555530000},"score":1,"resource":{"primary":{"URL":"https:\/\/peerj.com\/articles\/cs-720"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,11]]},"references-count":35,"alternative-id":["10.7717\/peerj-cs.720"],"URL":"https:\/\/doi.org\/10.7717\/peerj-cs.720","archive":["CLOCKSS","LOCKSS","Portico"],"relation":{"has-review":[{"id-type":"doi","id":"10.7287\/peerj-cs.720v0.3\/reviews\/1","asserted-by":"object"},{"id-type":"doi","id":"10.7287\/peerj-cs.720v0.1\/reviews\/1","asserted-by":"object"},{"id-type":"doi","id":"10.7287\/peerj-cs.720v0.1\/reviews\/2","asserted-by":"object"},{"id-type":"doi","id":"10.7287\/peerj-cs.720v0.2\/reviews\/2","asserted-by":"object"},{"id-type":"doi","id":"10.7287\/peerj-cs.720v0.2\/reviews\/1","asserted-by":"object"}]},"ISSN":["2376-5992"],"issn-type":[{"value":"2376-5992","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,11]]},"article-number":"e720"}}