{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T14:15:37Z","timestamp":1760710537855,"version":"3.41.0"},"reference-count":61,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2021,4,19]],"date-time":"2021-04-19T00:00:00Z","timestamp":1618790400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/P005888\/1"],"award-info":[{"award-number":["EP\/P005888\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2021,10,31]]},"abstract":"<jats:p>\n            Software changes constantly, because developers add new features or modifications. This directly affects the effectiveness of the test suite associated with that software, especially when these new modifications are in a specific area that no test case covers. This article tackles the problem of generating a high-quality test suite to cover repeatedly a given point in a program, with the ultimate goal of exposing faults possibly affecting the given program point. Both search-based software testing and constraint solving offer ready, but low-quality, solutions to this: Ideally, a\n            <jats:italic>maximally diverse<\/jats:italic>\n            covering test set is required, whereas search and constraint solving tend to generate test sets with biased distributions. Our approach, Diversified Focused Testing (DFT), uses a search strategy inspired by G\u00f6delTest. We artificially inject parameters into the code branching conditions and use a bi-objective search algorithm to find diverse inputs by perturbing the injected parameters, while keeping the path conditions still satisfiable. Our results demonstrate that our technique, DFT, is able to cover a desired point in the code at least 90% of the time. Moreover, adding diversity improves the bug detection and the mutation killing abilities of the test suites. We show that DFT achieves better results than focused testing, symbolic execution, and random testing by achieving from 3% to 70% improvement in mutation score and up to 100% improvement in fault detection across 105 software subjects.\n          <\/jats:p>","DOI":"10.1145\/3447265","type":"journal-article","created":{"date-parts":[[2021,4,19]],"date-time":"2021-04-19T23:40:21Z","timestamp":1618875621000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Diversifying Focused Testing for Unit Testing"],"prefix":"10.1145","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6314-3725","authenticated-orcid":false,"given":"H\u00e9ctor D.","family":"Men\u00e9ndez","sequence":"first","affiliation":[{"name":"Middlesex University London, Hendon, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gunel","family":"Jahangirova","sequence":"additional","affiliation":[{"name":"Universit\u00e0 della Svizzera italiana, Lugano, Svizzera"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Federica","family":"Sarro","sequence":"additional","affiliation":[{"name":"University College London, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paolo","family":"Tonella","sequence":"additional","affiliation":[{"name":"Universit\u00e0 della Svizzera italiana, Lugano, Svizzera"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Clark","sequence":"additional","affiliation":[{"name":"University College London, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,4,19]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Agitar. [n.d.]. Retrieved from https:\/\/www.agitar.com.  Agitar. [n.d.]. Retrieved from https:\/\/www.agitar.com."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931056"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.5555\/2337223.2337414"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2610384.2610413"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2013.02.061"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568314"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04772-5_38"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 8th International Workshop on Satisfiability Modulo Theories","volume":"13","author":"Barrett Clark","year":"2010","unstructured":"Clark Barrett , Aaron Stump , Cesare Tinelli et \u00a0al . 2010 . The smt-lib standard: Version 2.0 . In Proceedings of the 8th International Workshop on Satisfiability Modulo Theories , Vol. 13 . 14. Clark Barrett, Aaron Stump, Cesare Tinelli et\u00a0al. 2010. The smt-lib standard: Version 2.0. In Proceedings of the 8th International Workshop on Satisfiability Modulo Theories, Vol. 13. 14."},{"volume-title":"Proceedings of the International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS\u201915)","author":"Chakraborty Supratik","key":"e_1_2_1_9_1","unstructured":"Supratik Chakraborty , Daniel J. Fremont , Kuldeep S. Meel , Sanjit A. Seshia , and Moshe Y. Vardi . 2015. On parallel scalable uniform SAT witness generation . In Proceedings of the International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS\u201915) . 304--319. Supratik Chakraborty, Daniel J. Fremont, Kuldeep S. Meel, Sanjit A. Seshia, and Moshe Y. Vardi. 2015. On parallel scalable uniform SAT witness generation. In Proceedings of the International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS\u201915). 304--319."},{"key":"e_1_2_1_10_1","volume-title":"Vardi","author":"Chakraborty Supratik","year":"2016","unstructured":"Supratik Chakraborty , Kuldeep S. Meel , Rakesh Mistry , and Moshe Y . Vardi . 2016 . Approximate probabilistic inference via word-level counting. In Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI\u201916), Vol. 16 . 3218--3224. Supratik Chakraborty, Kuldeep S. Meel, Rakesh Mistry, and Moshe Y. Vardi. 2016. Approximate probabilistic inference via word-level counting. In Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI\u201916), Vol. 16. 3218--3224."},{"key":"e_1_2_1_11_1","volume-title":"Vardi","author":"Chakraborty Supratik","year":"2013","unstructured":"Supratik Chakraborty , Kuldeep S. Meel , and Moshe Y . Vardi . 2013 . A scalable approximate model counter. In Proceedings of the International Conference on Principles and Practice of Constraint Programming. Springer , 200--216. Supratik Chakraborty, Kuldeep S. Meel, and Moshe Y. Vardi. 2013. A scalable approximate model counter. In Proceedings of the International Conference on Principles and Practice of Constraint Programming. Springer, 200--216."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611973402.88"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/233269.233366"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2009.02.022"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.2307\/3315828"},{"key":"e_1_2_1_16_1","volume-title":"Thomas","author":"Cover Thomas M.","year":"2012","unstructured":"Thomas M. Cover and Joy A . Thomas . 2012 . Elements of Information Theory. John Wiley & Sons . Thomas M. Cover and Joy A. Thomas. 2012. Elements of Information Theory. John Wiley & Sons."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-78800-3_24"},{"key":"e_1_2_1_18_1","unstructured":"Ilias Diakonikolas Themis Gouleakis John Peebles and Eric Price. 2016. Collision-based testers are optimal for uniformity and closeness. Retrieved from https:\/\/arXiv:1611.03579.  Ilias Diakonikolas Themis Gouleakis John Peebles and Eric Price. 2016. Collision-based testers are optimal for uniformity and closeness. Retrieved from https:\/\/arXiv:1611.03579."},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the IEEE 24th International Symposium on Software Reliability Engineering (ISSRE\u201913)","author":"Feldt R.","year":"2013","unstructured":"R. Feldt and S. Poulding . 2013. Finding test data with specific properties via metaheuristic search . In Proceedings of the IEEE 24th International Symposium on Software Reliability Engineering (ISSRE\u201913) . 350--359. DOI:https:\/\/doi.org\/10.1109\/ISSRE. 2013 .6698888 10.1109\/ISSRE.2013.6698888 R. Feldt and S. Poulding. 2013. Finding test data with specific properties via metaheuristic search. In Proceedings of the IEEE 24th International Symposium on Software Reliability Engineering (ISSRE\u201913). 350--359. DOI:https:\/\/doi.org\/10.1109\/ISSRE.2013.6698888"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2016.33"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW.2008.36"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2006.1688440"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/QSIC.2011.19"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2025113.2025179"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2012.14"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2017.38"},{"volume-title":"Studies in Complexity and Cryptography. Miscellanea on the Interplay between Randomness and Computation","author":"Goldreich Oded","key":"e_1_2_1_27_1","unstructured":"Oded Goldreich and Dana Ron . 2011. On testing expansion in bounded-degree graphs . In Studies in Complexity and Cryptography. Miscellanea on the Interplay between Randomness and Computation . Springer , 68--75. Oded Goldreich and Dana Ron. 2011. On testing expansion in bounded-degree graphs. In Studies in Complexity and Cryptography. Miscellanea on the Interplay between Randomness and Computation. Springer, 68--75."},{"volume-title":"Advances In Neural Information Processing Systems","author":"Gomes Carla P.","key":"e_1_2_1_28_1","unstructured":"Carla P. Gomes , Ashish Sabharwal , and Bart Selman . 2007. Near-uniform sampling of combinatorial spaces using XOR constraints . In Advances In Neural Information Processing Systems . MIT Press , 481--488. Carla P. Gomes, Ashish Sabharwal, and Bart Selman. 2007. Near-uniform sampling of combinatorial spaces using XOR constraints. In Advances In Neural Information Processing Systems. MIT Press, 481--488."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2010.08.021"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/s100090050043"},{"key":"e_1_2_1_31_1","volume-title":"Wolfe","author":"Hollander Myles","year":"1999","unstructured":"Myles Hollander and Douglas A . Wolfe . 1999 . Nonparametric Statistical Methods. John Wiley & Sons . Myles Hollander and Douglas A. Wolfe. 1999. Nonparametric Statistical Methods. John Wiley & Sons."},{"key":"e_1_2_1_32_1","volume-title":"Proceedings of the Testing: Academic Industrial Conference\u2014Practice and Research Techniques (TAICPART\u201908)","author":"Jia Y.","year":"2008","unstructured":"Y. Jia and M. Harman . 2008. MILU: A customizable, runtime-optimized higher order mutation testing tool for the full C language . In Proceedings of the Testing: Academic Industrial Conference\u2014Practice and Research Techniques (TAICPART\u201908) . 94--98. DOI:https:\/\/doi.org\/10.1109\/TAIC-PART. 2008 .18 10.1109\/TAIC-PART.2008.18 Y. Jia and M. Harman. 2008. MILU: A customizable, runtime-optimized higher order mutation testing tool for the full C language. In Proceedings of the Testing: Academic Industrial Conference\u2014Practice and Research Techniques (TAICPART\u201908). 94--98. DOI:https:\/\/doi.org\/10.1109\/TAIC-PART.2008.18"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-66299-2_12"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-54862-8_26"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183399.3183405"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1080\/02331880500178562"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1555860.1555862"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2010.22"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2559932"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884797"},{"volume-title":"Proceedings of the 22nd ACM SIGPLAN Conference on Object-oriented Programming Systems and Applications Companion (OOPSLA\u201907)","author":"Pacheco Carlos","key":"e_1_2_1_41_1","unstructured":"Carlos Pacheco and Michael D. Ernst . 2007. Randoop: Feedback-directed random testing for Java . In Proceedings of the 22nd ACM SIGPLAN Conference on Object-oriented Programming Systems and Applications Companion (OOPSLA\u201907) . ACM, New York, NY, 815--816. DOI:https:\/\/doi.org\/10.1145\/1297846.1297902 10.1145\/1297846.1297902 Carlos Pacheco and Michael D. Ernst. 2007. Randoop: Feedback-directed random testing for Java. In Proceedings of the 22nd ACM SIGPLAN Conference on Object-oriented Programming Systems and Applications Companion (OOPSLA\u201907). ACM, New York, NY, 815--816. DOI:https:\/\/doi.org\/10.1145\/1297846.1297902"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.2307\/1402731"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2576768.2598339"},{"key":"e_1_2_1_44_1","volume-title":"R: A Language and Environment for Statistical Computing","author":"Team R Core","year":"2013","unstructured":"R Core Team . 2013 . R: A Language and Environment for Statistical Computing . R Foundation for Statistical Computing, Vienna, Austria . Retrieved from http:\/\/www.R-project.org\/. R Core Team. 2013. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from http:\/\/www.R-project.org\/."},{"key":"e_1_2_1_45_1","unstructured":"Gregg Rothermel Sebastian Elbaum Alex Kinneer and Hyunsook Do. 2006. Software-artifact infrastructure repository. Retrieved from http:\/\/sir. unl. edu\/portal.  Gregg Rothermel Sebastian Elbaum Alex Kinneer and Hyunsook Do. 2006. Software-artifact infrastructure repository. Retrieved from http:\/\/sir. unl. edu\/portal."},{"key":"e_1_2_1_46_1","volume-title":"Kroese","author":"Rubinstein Reuven Y.","year":"2016","unstructured":"Reuven Y. Rubinstein and Dirk P . Kroese . 2016 . Simulation and the Monte Carlo Method. Vol. 10 . John Wiley & Sons . Reuven Y. Rubinstein and Dirk P. Kroese. 2016. Simulation and the Monte Carlo Method. Vol. 10. John Wiley & Sons."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2650914"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884830"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2010.38"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/1321631.1321746"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2015.86"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2015.2434953"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.5555\/1929004.1929011"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2018.8330203"},{"key":"e_1_2_1_55_1","volume-title":"Proceedings of the International Congress on Modelling and Simulation. Advances and Applications for Management and Decision Making (MODSIM\u201905)","author":"Steele Mike","year":"2005","unstructured":"Mike Steele , Janet Chaseling , and Cameron Hurst . 2005 . Simulated power of the discrete Cram\u00e9r-von Mises goodness-of-fit tests . In Proceedings of the International Congress on Modelling and Simulation. Advances and Applications for Management and Decision Making (MODSIM\u201905) . 1300--1304. Mike Steele, Janet Chaseling, and Cameron Hurst. 2005. Simulated power of the discrete Cram\u00e9r-von Mises goodness-of-fit tests. In Proceedings of the International Congress on Modelling and Simulation. Advances and Applications for Management and Decision Making (MODSIM\u201905). 1300--1304."},{"key":"e_1_2_1_56_1","volume-title":"Proceedings of the 39th International Conference on Software Engineering Companion. IEEE Press, 180--182","author":"Tan Shin Hwei","year":"2017","unstructured":"Shin Hwei Tan , Jooyong Yi , Sergey Mechtaev , Abhik Roychoudhury et \u00a0al . 2017 . Codeflaws: A programming competition benchmark for evaluating automated program repair tools . In Proceedings of the 39th International Conference on Software Engineering Companion. IEEE Press, 180--182 . Shin Hwei Tan, Jooyong Yi, Sergey Mechtaev, Abhik Roychoudhury et\u00a0al. 2017. Codeflaws: A programming competition benchmark for evaluating automated program repair tools. In Proceedings of the 39th International Conference on Software Engineering Companion. IEEE Press, 180--182."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2009.5070507"},{"key":"e_1_2_1_58_1","volume-title":"Proceedings of the Network and Distributed System Security Symposium (NDSS\u201916)","volume":"16","author":"Michelle","unstructured":"Michelle Y. Wong and David Lie. 2016. IntelliDroid: A targeted input generator for the dynamic analysis of android malware . In Proceedings of the Network and Distributed System Security Symposium (NDSS\u201916) , Vol. 16 . 21--24. Michelle Y. Wong and David Lie. 2016. IntelliDroid: A targeted input generator for the dynamic analysis of android malware. In Proceedings of the Network and Distributed System Security Symposium (NDSS\u201916), Vol. 16. 21--24."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9552-y"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1137\/0218082"},{"key":"e_1_2_1_61_1","volume-title":"SPEA2: Improving the strength Pareto evolutionary algorithm. TIK-report 103","author":"Zitzler Eckart","year":"2001","unstructured":"Eckart Zitzler , Marco Laumanns , and Lothar Thiele . 2001. SPEA2: Improving the strength Pareto evolutionary algorithm. TIK-report 103 ( 2001 ). Eckart Zitzler, Marco Laumanns, and Lothar Thiele. 2001. SPEA2: Improving the strength Pareto evolutionary algorithm. TIK-report 103 (2001)."}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447265","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3447265","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:28:27Z","timestamp":1750195707000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447265"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,19]]},"references-count":61,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,10,31]]}},"alternative-id":["10.1145\/3447265"],"URL":"https:\/\/doi.org\/10.1145\/3447265","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"type":"print","value":"1049-331X"},{"type":"electronic","value":"1557-7392"}],"subject":[],"published":{"date-parts":[[2021,4,19]]},"assertion":[{"value":"2019-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-04-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}