{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T08:34:42Z","timestamp":1772872482264,"version":"3.50.1"},"reference-count":16,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2005,2,1]],"date-time":"2005-02-01T00:00:00Z","timestamp":1107216000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Queue"],"published-print":{"date-parts":[[2005,2]]},"abstract":"<jats:p>The increasing size and complexity of software, coupled with concurrency and distributed systems, has made apparent the ineffectiveness of using only handcrafted tests. The misuse of code coverage and avoidance of random testing has exacerbated the problem. We must start again, beginning with good design (including dependency analysis), good static checking (including model property checking), and good unit testing (including good input selection). Code coverage can help select and prioritize tests to make you more efficient, as can the all-pairs technique for controlling the number of configurations. Finally, testers can use models to generate test coverage and good stochastic tests, and to act as test oracles.<\/jats:p>","DOI":"10.1145\/1046931.1046944","type":"journal-article","created":{"date-parts":[[2005,8,3]],"date-time":"2005-08-03T08:30:55Z","timestamp":1123057855000},"page":"30-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Too Darned Big to Test"],"prefix":"10.1145","volume":"3","author":[{"given":"Keith","family":"Stobie","sequence":"first","affiliation":[{"name":"Microsoft"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2005,2]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Kaner C. 2000. Architectures of test automation. http:\/\/www.kaner.com\/testarch.html.  Kaner C. 2000. Architectures of test automation. http:\/\/www.kaner.com\/testarch.html."},{"key":"e_1_2_1_2_1","unstructured":"Hoffman D. So little time so many cases. www.cs.bsu. edu\/homepages\/dmz\/cs639\/So%20little%20time %20so%20many%20cases.ppt.  Hoffman D. So little time so many cases. www.cs.bsu. edu\/homepages\/dmz\/cs639\/So%20little%20time %20so%20many%20cases.ppt."},{"key":"e_1_2_1_3_1","unstructured":"Howard M. and LeBlanc D. 2002. Writing secure code 2nd edition. Microsoft Press.   Howard M. and LeBlanc D. 2002. Writing secure code 2nd edition. Microsoft Press."},{"key":"e_1_2_1_4_1","first-page":"165","article-title":"A statistical assessment of some software testing strategies and application of experimental design techniques","volume":"8","author":"Nair V. N.","year":"1998","journal-title":"Statistica Sinica"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.962563"},{"key":"e_1_2_1_6_1","volume-title":"22nd Annual Pacific Northwest Software Quality Conference","author":"Bach J.","year":"2004"},{"key":"e_1_2_1_7_1","unstructured":"Hoffman D. 1999. Heuristic test oracles. Software Testing and Quality Engineering (March\/April). http:\/\/softwarequalitymethods.com\/SQM\/Papers\/HeuristicPaper.pdf.  Hoffman D. 1999. Heuristic test oracles. Software Testing and Quality Engineering (March\/April). http:\/\/softwarequalitymethods.com\/SQM\/Papers\/HeuristicPaper.pdf."},{"key":"e_1_2_1_8_1","volume-title":"Quality Week '98","author":"Hoffman D.","year":"1998"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/996821.996838"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2004.1293079"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/265924.265927"},{"key":"e_1_2_1_12_1","unstructured":"Tate A. 2003. The best testers are free. Software Testing Analysis and Review (STAR West).  Tate A. 2003. The best testers are free. Software Testing Analysis and Review (STAR West)."},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of the 16th IEEE Conference on Automated Software Engineering (November): 22-31","author":"Marinov D."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1002\/1099-1689(200009)10:3<149::AID-STVR206>3.0.CO;2-T"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.5555\/590565.590594"},{"key":"e_1_2_1_16_1","unstructured":"Robinson H. and Corning M. Model-based testing in the key of C#. http:\/\/www.qasig.org\/presentations\/QASIG_Goldilocks2.pdf.  Robinson H. and Corning M. Model-based testing in the key of C#. http:\/\/www.qasig.org\/presentations\/QASIG_Goldilocks2.pdf."}],"container-title":["Queue"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1046931.1046944","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1046931.1046944","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T16:24:57Z","timestamp":1750263897000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1046931.1046944"}},"subtitle":["Testing large systems is a daunting task, but there are steps we can take to ease the pain."],"short-title":[],"issued":{"date-parts":[[2005,2]]},"references-count":16,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2005,2]]}},"alternative-id":["10.1145\/1046931.1046944"],"URL":"https:\/\/doi.org\/10.1145\/1046931.1046944","relation":{},"ISSN":["1542-7730","1542-7749"],"issn-type":[{"value":"1542-7730","type":"print"},{"value":"1542-7749","type":"electronic"}],"subject":[],"published":{"date-parts":[[2005,2]]},"assertion":[{"value":"2005-02-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}