{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T18:40:10Z","timestamp":1784572810277,"version":"3.55.0"},"reference-count":63,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2013,2,1]],"date-time":"2013-02-01T00:00:00Z","timestamp":1359676800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2013,2]]},"abstract":"<jats:p>The increase in size and complexity of modern software systems requires scalable, systematic, and automated testing approaches. Model-based testing (MBT), as a systematic and automated test case generation technique, is being successfully applied to verify industrial-scale systems and is supported by commercial tools. However, scalability is still an open issue for large systems, as in practice there are limits to the amount of testing that can be performed in industrial contexts. Even with standard coverage criteria, the resulting test suites generated by MBT techniques can be very large and expensive to execute, especially for system level testing on real deployment platforms and network facilities. Therefore, a scalable MBT technique should be flexible regarding the size of the generated test suites and should be easily accommodated to fit resource and time constraints. Our approach is to select a subset of the generated test suite in such a way that it can be realistically executed and analyzed within the time and resource constraints, while preserving the fault revealing power of the original test suite to a maximum extent. In this article, to address this problem, we introduce a family of similarity-based test case selection techniques for test suites generated from state machines. We evaluate 320 different similarity-based selection techniques and then compare the effectiveness of the best similarity-based selection technique with other common selection techniques in the literature. The results based on two industrial case studies, in the domain of embedded systems, show significant benefits and a large improvement in performance when using a similarity-based approach. We complement these analyses with further studies on the scalability of the technique and the effects of failure rate on its effectiveness. We also propose a method to identify optimal tradeoffs between the number of test cases to run and fault detection.<\/jats:p>","DOI":"10.1145\/2430536.2430540","type":"journal-article","created":{"date-parts":[[2013,3,5]],"date-time":"2013-03-05T20:28:36Z","timestamp":1362515316000},"page":"1-42","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":157,"title":["Achieving scalable model-based testing through test case diversity"],"prefix":"10.1145","volume":"22","author":[{"given":"Hadi","family":"Hemmati","sequence":"first","affiliation":[{"name":"Simula Research Laboratory, Norway and University of Oslo, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"Arcuri","sequence":"additional","affiliation":[{"name":"Simula Research Laboratory, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lionel","family":"Briand","sequence":"additional","affiliation":[{"name":"Simula Research Laboratory, Norway and University of Oslo, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2013,3,4]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Aarts E. H. L. and Lenstra J. K. 2003. Local Search in Combinatorial Optimization. Princeton University Press.   Aarts E. H. L. and Lenstra J. K. 2003. Local Search in Combinatorial Optimization. Princeton University Press.","DOI":"10.1515\/9780691187563"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2009.52"},{"key":"e_1_2_1_3_1","first-page":"2010","article-title":"Model transformations as a strategy to automate model-based testing: A tool and industrial case studies Simula Research Laboratory","author":"Ali S.","year":"2010","unstructured":"Ali , S. , Hemmati , H. , Holt , N. E. , Arisholm , E. , and Briand , L. 2010 b. Model transformations as a strategy to automate model-based testing: A tool and industrial case studies Simula Research Laboratory , Tech. Rep. 2010 - 2001 . Ali, S., Hemmati, H., Holt, N. E., Arisholm, E., and Briand, L. 2010b. Model transformations as a strategy to automate model-based testing: A tool and industrial case studies Simula Research Laboratory, Tech. Rep. 2010-01.","journal-title":"Tech. Rep."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2010.16"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985795"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2007.11.024"},{"key":"e_1_2_1_7_1","volume-title":"Testing Object-Oriented Systems: Models, Patterns, and Tools","author":"Binder R. V.","unstructured":"Binder , R. V. 1999. Testing Object-Oriented Systems: Models, Patterns, and Tools . Addison-Wesley Professional . Binder, R. V. 1999. Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison-Wesley Professional."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.413"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2009.02.022"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1002\/smr.v21:6"},{"key":"e_1_2_1_11_1","unstructured":"Cormen T. H. Leiserson C. E. Rivest R. L. and Stein C. 2001. Introduction to Algorithms. MIT Press.   Cormen T. H. Leiserson C. E. Rivest R. L. and Stein C. 2001. Introduction to Algorithms. MIT Press."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1062455.1062606"},{"key":"e_1_2_1_13_1","unstructured":"Dong G. and Pei J. 2007. Sequence Data Mining. Springer.   Dong G. and Pei J. 2007. Sequence Data Mining. Springer."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0304-3975(01)00182-7"},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","unstructured":"Durbin R. Eddy S. R. Krogh A. and Mitchison G. 1999. Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press.  Durbin R. Eddy S. R. Krogh A. and Mitchison G. 1999. Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press.","DOI":"10.1017\/CBO9780511790492"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.988497"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW.2008.36"},{"key":"e_1_2_1_18_1","volume-title":"An Introduction to Probability Theory and Its Applications","author":"Feller W.","unstructured":"Feller , W. 1968. An Introduction to Probability Theory and Its Applications . Wiley . Feller, W. 1968. An Introduction to Probability Theory and Its Applications. Wiley."},{"key":"e_1_2_1_19_1","volume-title":"Design Patterns: Elements of Reusable Object-Oriented Software","author":"Gamma E.","year":"1995","unstructured":"Gamma , E. , Helm , R. , Johnson , R. , and Vlissides , J . 1995 . Design Patterns: Elements of Reusable Object-Oriented Software . Addison-Wesley . Gamma, E., Helm, R., Johnson, R., and Vlissides, J. 1995. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley."},{"key":"e_1_2_1_20_1","volume-title":"Genetic Algorithms in Search, Optimization, and Machine Learning","author":"Goldberg D. E.","unstructured":"Goldberg , D. E. 2001. Genetic Algorithms in Search, Optimization, and Machine Learning . Addison-Wesley Professional . Goldberg, D. E. 2001. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley Professional."},{"key":"e_1_2_1_21_1","volume-title":"Algorithms on Strings, Trees and Sequences: Computer Science and Computational Biology","author":"Gusfield D.","unstructured":"Gusfield , D. 1997. Algorithms on Strings, Trees and Sequences: Computer Science and Computational Biology . Cambridge University Press . Gusfield, D. 1997. Algorithms on Strings, Trees and Sequences: Computer Science and Computational Biology. Cambridge University Press."},{"key":"e_1_2_1_22_1","volume-title":"Encyclopedia of Software Engineering","author":"Hamlet R.","unstructured":"Hamlet , R. 1994. Random testing . In Encyclopedia of Software Engineering . Wiley , 970--978. Hamlet, R. 1994. Random testing. In Encyclopedia of Software Engineering. Wiley, 970--978."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/FOSE.2007.29"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0950-5849(01)00189-6"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2009.71"},{"key":"e_1_2_1_26_1","unstructured":"Haupt R. L. and Haupt S. E. 1998. Practical Genetic Algorithms. Wiley-Interscience.   Haupt R. L. and Haupt S. E. 1998. Practical Genetic Algorithms. Wiley-Interscience."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/1928028.1928034"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2011.12"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2010.9"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1882291.1882331"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2009.09.011"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2009.77"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2003.1183927"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/11902140_106"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1291535.1291539"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276958.1277175"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2009.23"},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the 14th IEEE International Symposium on Software Reliability Engineering. 442--456","author":"Leon D.","unstructured":"Leon , D. and Podgurski , A . 2003. A comparison of coverage-based and distribution-based techniques for filtering and prioritizing test cases . In Proceedings of the 14th IEEE International Symposium on Software Reliability Engineering. 442--456 . Leon, D. and Podgurski, A. 2003. A comparison of coverage-based and distribution-based techniques for filtering and prioritizing test cases. In Proceedings of the 14th IEEE International Symposium on Software Reliability Engineering. 442--456."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.38"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/11573937_28"},{"key":"e_1_2_1_41_1","doi-asserted-by":"crossref","unstructured":"Manning C. D. Raghavan P. and Sch\u00fctze H. 2008. Introduction to Information Retrieval. Cambridge University Press.   Manning C. D. Raghavan P. and Sch\u00fctze H. 2008. Introduction to Information Retrieval. Cambridge University Press.","DOI":"10.1017\/CBO9780511809071"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.1020"},{"key":"e_1_2_1_43_1","volume-title":"Foundations of Software Testing","author":"Mathur A. P.","unstructured":"Mathur , A. P. 2008. Foundations of Software Testing . Addison-Wesley Professional . Mathur, A. P. 2008. Foundations of Software Testing. Addison-Wesley Professional."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.70756"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1162\/evco.1996.4.1.1"},{"key":"e_1_2_1_46_1","doi-asserted-by":"crossref","unstructured":"Moscato P. and Cotta C. 2010. A modern introduction to memetic algorithms. In Handbook of Metaheuristics. Springer 141--183.  Moscato P. and Cotta C. 2010. A modern introduction to memetic algorithms. In Handbook of Metaheuristics. Springer 141--183.","DOI":"10.1007\/978-1-4419-1665-5_6"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.v17:2"},{"key":"e_1_2_1_48_1","unstructured":"Pender T. 2003. UML Bible. Wiley.   Pender T. 2003. UML Bible. Wiley."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/1363686.1363848"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.256"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2006.103"},{"key":"e_1_2_1_52_1","unstructured":"Tan P. N. Steinbach M. and Kumar V. 2006. Introduction to Data Mining. Addison-Wesley.   Tan P. N. Steinbach M. and Kumar V. 2006. Introduction to Data Mining. Addison-Wesley."},{"key":"e_1_2_1_53_1","unstructured":"Utting M. and Legeard B. 2006. Practical Model-Based Testing: A Tools Approach. Morgan-Kaufmann.   Utting M. and Legeard B. 2006. Practical Model-Based Testing: A Tools Approach. Morgan-Kaufmann."},{"key":"e_1_2_1_54_1","volume-title":"Tech.Rep. ICIS-R06019","author":"Vaandrager F. W.","unstructured":"Vaandrager , F. W. 2006. Does it pay off&quest ; Model-based verification and validation of embedded systems. Tech.Rep. ICIS-R06019 , ICIS Radboud University Nijmegen , 43--66. Vaandrager, F. W. 2006. Does it pay off&quest; Model-based verification and validation of embedded systems. Tech.Rep. ICIS-R06019, ICIS Radboud University Nijmegen, 43--66."},{"key":"e_1_2_1_55_1","first-page":"101","article-title":"A critique and improvement of the CL common language effect size statistics of McGraw and Wong","volume":"25","author":"Vargha A.","year":"2000","unstructured":"Vargha , A. and Delaney , H. D. 2000 . A critique and improvement of the CL common language effect size statistics of McGraw and Wong . J. Educ. Behav. Stat. 25 , 101 -- 132 . Vargha, A. and Delaney, H. D. 2000. A critique and improvement of the CL common language effect size statistics of McGraw and Wong. J. Educ. Behav. Stat. 25, 101--132.","journal-title":"J. Educ. Behav. Stat."},{"key":"e_1_2_1_56_1","volume-title":"Proceedings of the 19th IFIP International Conference on Testing of Communicating Systems and 7th International Workshop on Formal Approaches to Testing of Software. 351--364","author":"Vega D.","unstructured":"Vega , D. , Schieferdecker , I. , and Din , G . 2007. Test data variance as a test quality measure: Exemplified for TTCN-3 . In Proceedings of the 19th IFIP International Conference on Testing of Communicating Systems and 7th International Workshop on Formal Approaches to Testing of Software. 351--364 . Vega, D., Schieferdecker, I., and Din, G. 2007. Test data variance as a test quality measure: Exemplified for TTCN-3. In Proceedings of the 19th IFIP International Conference on Testing of Communicating Systems and 7th International Workshop on Formal Approaches to Testing of Software. 351--364."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1980.234486"},{"key":"e_1_2_1_58_1","volume-title":"Proceedings of the 3rd International Conference on Genetic Algorithms. 116--121","author":"Whitley D.","year":"1989","unstructured":"Whitley , D. 1989 . The GENITOR algorithm and selection pressure: Why rank-based allocation of reproductive trials is best . In Proceedings of the 3rd International Conference on Genetic Algorithms. 116--121 . Whitley, D. 1989. The GENITOR algorithm and selection pressure: Why rank-based allocation of reproductive trials is best. In Proceedings of the 3rd International Conference on Genetic Algorithms. 116--121."},{"key":"e_1_2_1_59_1","volume-title":"Software Engineering: An Introduction","author":"Wohlin C.","year":"2000","unstructured":"Wohlin , C. , Runeson , P. , Host , M. , Ohlsson , M. C. , Regnell , B. , and Wesslen , A . 2000 . Experimentation in Software Engineering: An Introduction . Kluwer Academic Publishers . Wohlin, C., Runeson, P., Host, M., Ohlsson, M. C., Regnell, B., and Wesslen, A. 2000. Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/4235.585893"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-006-9026-0"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2005.845141"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/1572272.1572296"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2430536.2430540","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2430536.2430540","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T08:35:23Z","timestamp":1750235723000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2430536.2430540"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,2]]},"references-count":63,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2013,2]]}},"alternative-id":["10.1145\/2430536.2430540"],"URL":"https:\/\/doi.org\/10.1145\/2430536.2430540","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2013,2]]},"assertion":[{"value":"2010-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2011-10-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-03-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}