{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T08:03:11Z","timestamp":1772524991722,"version":"3.50.1"},"reference-count":30,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2006,10,31]],"date-time":"2006-10-31T00:00:00Z","timestamp":1162252800000},"content-version":"vor","delay-in-days":4686,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Software Testing Verif &amp; Rel"],"published-print":{"date-parts":[[1994,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Evaluation of the adequacy of a test set consisting of one or more test cases is a problem oftes encountered in software testing environments. Two test adequacy criiteria are considered, namely the data flow based all\u2010uses criterion and a mutation based criterion. An empirical study was conducted to compare the \u2018difficulty\u2019 of satisfying the two criteria and their costs. Similar studies conducted in the past are discussed in the light of this study. A discussion is also presented of how and why the results of this study, when viewed in conjunction with the results of earlier comparisons of testing methods, are useful to a software test team.<\/jats:p>","DOI":"10.1002\/stvr.4370040104","type":"journal-article","created":{"date-parts":[[2006,11,18]],"date-time":"2006-11-18T06:12:24Z","timestamp":1163830344000},"page":"9-31","source":"Crossref","is-referenced-by-count":85,"title":["An empirical comparison of data flow and mutation\u2010based test adequacy criteria"],"prefix":"10.1002","volume":"4","author":[{"given":"Aditya P.","family":"Mathur","sequence":"first","affiliation":[]},{"given":"W. Eric","family":"Wong","sequence":"additional","affiliation":[]}],"member":"311","published-online":{"date-parts":[[2006,10,31]]},"reference":[{"key":"e_1_2_1_2_1","unstructured":"Acree A. T.(1980) \u2018On mutation\u2019 Ph. D. thesis School of Information and Computer Science Georgia Institute of Technology Atlanta Georgia U.S.A. Technical Report GIT\u2010ICS\u201080\/12."},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Acree A. T. Budd T. A. DeMillo R. A. Lipton R. J.andSayward F. G.(1979) \u2018Mutation analysis\u2019 Technical Report GIT\u2010ICS\u201079\/08 School of Information and Computer Science Georgia Institute of Technology Atlanta Georgia U.S.A.","DOI":"10.21236\/ADA076575"},{"key":"e_1_2_1_4_1","unstructured":"Agrawal H. DeMillo R. A. Hathaway R. Hsu Wm. Hsu Wy. Krauser E. W. Martin R. J. Mathur A. P.andSpafford E. H.(1989) \u2018Design of mutant operators for the C programming language\u2019 Technical Report SERC\u2010TR\u201041\u2013P Software Engineering Research Center Purdue University West Lafayette Indiana U.S.A."},{"key":"e_1_2_1_5_1","unstructured":"Budd T. A.(1980) \u2018Mutation analysis of program test data\u2019. Ph. D. thesis Yale University New Haven Connecticut U.S.A."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/0164-1212(93)90005-I"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.41326"},{"issue":"4","key":"e_1_2_1_8_1","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1109\/C-M.1978.218136","article-title":"Hints on test data selection: Help for the practicing programmer","volume":"11","author":"DeMillo R. A.","year":"1978","journal-title":"IEEE Computer"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.92910"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1984.5010257"},{"key":"e_1_2_1_11_1","unstructured":"Frankl P. G.andWeiss S. N.(1991) \u2018Is data flow testing more effective than branch testing? An empirical study \u2019 in Proceedings Quality Week 1991 San Francisco California U.S.A."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1975.6312836"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1983.235433"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.62448"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/362452.362489"},{"key":"e_1_2_1_16_1","unstructured":"Horgan J. R.andLondon S. A.(1990) \u2018ATAC\u2010automatic test analysis for C programs \u2019 Internal memorandum TM\u2010TSV\u2010017980 Bell Communications Research Morristown New Jersey U.S.A."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/120807.120815"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/52.136170"},{"key":"e_1_2_1_19_1","unstructured":"Mathur A. P.(1991a ) \u2018On the relative strengths of data flow and mutation testing\u2019 InProceedings of the Ninth Annual Pacific Northwest Software Quality Conference Portland Oregon U.S.A."},{"key":"e_1_2_1_20_1","doi-asserted-by":"crossref","unstructured":"Mathur A. P.(1991b ) \u2018Performance effectiveness and reliability issues in software testing\u2019 InProceedings of the Fifteenth Annual International Computer Software and Applications Conference Tokyo Japan pp.604\u2013605.","DOI":"10.1109\/CMPSAC.1991.170248"},{"key":"e_1_2_1_21_1","unstructured":"Mathur A. P.andWong W. E.(1993a) \u2018An empirical comparison of mutation and data flow based test adequacy criteria \u2019 In Proceedings of the Software Engineering Research Forum 93 (SERF\u201093) Orlando Florida U.S.A. (also available as SERC\u2010TR\u2010135\u2010P. Software Engineering Research Center Purdue University West Lafayette Indiana U.S.A.)."},{"key":"e_1_2_1_22_1","unstructured":"Mathur A. P.andWong W. E.(1993b ) \u2018Comparing the error detection effectiveness of mutation and data flow testing: An empirical study\u2019 Technical Report SERC\u2010TR\u2010146\u2010P Software Engineering Research Center Purdue University West Lafayette Indiana U.S.A."},{"key":"e_1_2_1_23_1","unstructured":"Mathur A. P.andWong W. E.(1993c) \u2018Evaluation of the cost of alternate mutation testing strategies \u2019 In Proceedings of the Seventh Brazilian Symposium on Software Engineering Rio de Janiero Brazil (also available as SERC\u2010TR\u2010138\u2010P Software Engineering Research Center Purdue University West Lafayette Indiana U.S.A.)."},{"key":"e_1_2_1_24_1","doi-asserted-by":"crossref","unstructured":"Mathur A. P.andWong W. E.(1994) \u2018A theoretical comparison between mutation and data flow based test adequacy criteria\u2019 InProceedings of 1994 ACM Computer Science Conference (CSC\u203294)(also available as SERC\u2010TR\u2010133\u2010P Software Engineering Research Center Purdue University West Lafayette Indiana U.S.A.).","DOI":"10.1145\/197530.197554"},{"key":"e_1_2_1_25_1","unstructured":"Offutt A. J. Rothermel G.andZapf C.(1993) \u2018An experimental evaluation of selective mutation\u2019 InProceedings of the 15th International Conference on Software Engineering.Baltimore Maryland U.S.A. IEEE Computer Society Press pp.100\u2013107."},{"key":"e_1_2_1_26_1","unstructured":"Ohba M.andBasili V. R.(1991) \u2018A study of large scale software testing.\u2019 In Proceedings of the 16th Annual Software Engineering Workshop (SEL\u201091\u2013006 NASA GSFC) Greenbelt Maryland U.S.A. pp.199\u2013207."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1985.232226"},{"key":"e_1_2_1_28_1","first-page":"149","volume-title":"Proceedings of the International Symposium on Software Testing and Analysis (ISSTA)","author":"Weiss S. N.","year":"1993"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/0020-0190(84)90106-6"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.44376"},{"key":"e_1_2_1_31_1","first-page":"154","volume-title":"Proceedings of the Fourth Symposium on Software Testing, Analysis and Verification","author":"Weyuker E. J.","year":"1991"}],"container-title":["Software Testing, Verification and Reliability"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fstvr.4370040104","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/stvr.4370040104","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,12]],"date-time":"2025-01-12T00:06:14Z","timestamp":1736640374000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/stvr.4370040104"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[1994,1]]},"references-count":30,"journal-issue":{"issue":"1","published-print":{"date-parts":[[1994,1]]}},"alternative-id":["10.1002\/stvr.4370040104"],"URL":"https:\/\/doi.org\/10.1002\/stvr.4370040104","archive":["Portico"],"relation":{},"ISSN":["0960-0833","1099-1689"],"issn-type":[{"value":"0960-0833","type":"print"},{"value":"1099-1689","type":"electronic"}],"subject":[],"published":{"date-parts":[[1994,1]]}}}