{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,20]],"date-time":"2025-11-20T19:01:29Z","timestamp":1763665289709,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":23,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,9,25]],"date-time":"2023-09-25T00:00:00Z","timestamp":1695600000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Conselho Nacional de Desenvolvimento Cient\u00edfico e Tecnologico","award":["308445\/2021-0"],"award-info":[{"award-number":["308445\/2021-0"]}]},{"name":"Fundacao de Amparo \u00e0 Pesquisa do Estado de S\u00e3o Paulo","award":["2019\/06937-0"],"award-info":[{"award-number":["2019\/06937-0"]}]},{"name":"Conselho Nacional de Desenvolvimento Cient\u00edfico e Tecnol\u00f3gico","award":["308636\/2021-0"],"award-info":[{"award-number":["308636\/2021-0"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,9,25]]},"DOI":"10.1145\/3624032.3624038","type":"proceedings-article","created":{"date-parts":[[2023,10,17]],"date-time":"2023-10-17T18:15:18Z","timestamp":1697566518000},"page":"38-46","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Test Data Selection Based on Applying Mutation Testing to Decision Tree Models"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-1507-3367","authenticated-orcid":false,"given":"Beatriz","family":"Silveira","sequence":"first","affiliation":[{"name":"Departamento de Sistemas de Computa\u00e7\u00e3o, Universidade de S\u00e3o Paulo, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5768-1850","authenticated-orcid":false,"given":"Vinicius","family":"Durelli","sequence":"additional","affiliation":[{"name":"Universidade Federal de S\u00e3o Jo\u00e3o del Rei, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2797-7079","authenticated-orcid":false,"given":"Sebasti\u00e3o","family":"Santos","sequence":"additional","affiliation":[{"name":"Departamento de Sistemas de Computa\u00e7\u00e3o, Universidade de S\u00e3o Paulo, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6343-7715","authenticated-orcid":false,"given":"Rafael","family":"Durelli","sequence":"additional","affiliation":[{"name":"Universidade Federal de Lavras - UFLA, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7535-5891","authenticated-orcid":false,"given":"Marcio","family":"Delamaro","sequence":"additional","affiliation":[{"name":"Departamento de Sistemas de Computa\u00e7\u00e3o, Universidade de S\u00e3o Paulo, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9007-9821","authenticated-orcid":false,"given":"Simone","family":"Souza","sequence":"additional","affiliation":[{"name":"Departamento de Sistemas de Computa\u00e7\u00e3o, ICMC-USP, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,10,17]]},"reference":[{"volume-title":"IN 47907","author":"Agrawal Hiralal","key":"e_1_3_2_1_1_1","unstructured":"Hiralal Agrawal , Richard\u00a0 A. Demillo , Bob Hathaway , William Hsu , Wynne Hsu , E.\u00a0 W. Krauser , R.\u00a0 J. Martin , Aditya\u00a0 P. Mathur , and Eugene\u00a0 H. Spafford . 1989. Design Of Mutant Operators For The C Programming Language. W. Lafayette , IN 47907 , Software Engineering Research Center Department of Computer Sciences Purdue University . Hiralal Agrawal, Richard\u00a0A. Demillo, Bob Hathaway, William Hsu, Wynne Hsu, E.\u00a0W. Krauser, R.\u00a0J. Martin, Aditya\u00a0P. Mathur, and Eugene\u00a0H. Spafford. 1989. Design Of Mutant Operators For The C Programming Language. W. Lafayette, IN 47907, Software Engineering Research Center Department of Computer Sciences Purdue University."},{"volume-title":"Introduction to software testing","author":"Ammann Paul","key":"e_1_3_2_1_2_1","unstructured":"Paul Ammann and Jeff Offutt . 2016. Introduction to software testing . Cambridge University Press . Paul Ammann and Jeff Offutt. 2016. Introduction to software testing. Cambridge University Press."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2020.3021736"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110542"},{"key":"e_1_3_2_1_5_1","first-page":"34","article-title":"Hints on Test Data Selection","volume":"11","author":"DeMillo R.A.","year":"1978","unstructured":"R.A. DeMillo , R.J. Lipton , and F.G. Sayward . 1978 . Hints on Test Data Selection : Help for the Practicing Programmer. Computer 11 , 4 (1978), 34 \u2013 41 . R.A. DeMillo, R.J. Lipton, and F.G. Sayward. 1978. Hints on Test Data Selection: Help for the Practicing Programmer. Computer 11, 4 (1978), 34\u201341.","journal-title":"Help for the Practicing Programmer. Computer"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"crossref","unstructured":"V.\u00a0H.\u00a0S. Durelli R.\u00a0S. Durelli S.\u00a0S. Borges A.\u00a0T. Endo M.\u00a0M. Eler D.\u00a0R.\u00a0C. Dias and M.\u00a0P. Guimar\u00e3es. 2019. Machine Learning Applied to Software Testing: A Systematic Mapping Study.  V.\u00a0H.\u00a0S. Durelli R.\u00a0S. Durelli S.\u00a0S. Borges A.\u00a0T. Endo M.\u00a0M. Eler D.\u00a0R.\u00a0C. Dias and M.\u00a0P. Guimar\u00e3es. 2019. Machine Learning Applied to Software Testing: A Systematic Mapping Study.","DOI":"10.1109\/TR.2019.2892517"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1469-1809.1936.tb02137.x"},{"key":"e_1_3_2_1_8_1","volume-title":"Hands-On Machine Learning with Scikit-Learn, Keras, and Tensorflow: Concepts, Tools, and Techniques to Build Intelligent Systems","author":"G\u00e9ron Aur\u00e9lien","unstructured":"Aur\u00e9lien G\u00e9ron . 2019. Hands-On Machine Learning with Scikit-Learn, Keras, and Tensorflow: Concepts, Tools, and Techniques to Build Intelligent Systems ( 2 nd ed.). O\u2019Reilly . 856 pages. Aur\u00e9lien G\u00e9ron. 2019. Hands-On Machine Learning with Scikit-Learn, Keras, and Tensorflow: Concepts, Tools, and Techniques to Build Intelligent Systems (2nd ed.). O\u2019Reilly. 856 pages.","edition":"2"},{"volume-title":"34th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 1157\u20131161","author":"Hu Q.","key":"e_1_3_2_1_9_1","unstructured":"Q. Hu , L. Ma , X. Xie , B. Yu , Y. Liu , and J. Zhao . 2019. DeepMutation++: A Mutation Testing Framework for Deep Learning Systems . In 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 1157\u20131161 . Q. Hu, L. Ma, X. Xie, B. Yu, Y. Liu, and J. Zhao. 2019. DeepMutation++: A Mutation Testing Framework for Deep Learning Systems. In 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 1157\u20131161."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3460319.3464825"},{"key":"e_1_3_2_1_11_1","volume-title":"An Empirical Evaluation of Mutation Operators for Deep Learning Systems. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST). 74\u201384","author":"Jahangirova Gunel","year":"2020","unstructured":"Gunel Jahangirova and Paolo Tonella . 2020 . An Empirical Evaluation of Mutation Operators for Deep Learning Systems. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST). 74\u201384 . Gunel Jahangirova and Paolo Tonella. 2020. An Empirical Evaluation of Mutation Operators for Deep Learning Systems. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST). 74\u201384."},{"volume-title":"An Introduction to Statistical Learning: with Applications in R","author":"James Gareth","key":"e_1_3_2_1_12_1","unstructured":"Gareth James , Daniela Witten , Trevor Hastie , and Robert Tibshirani . 2013. An Introduction to Statistical Learning: with Applications in R . Springer Texts in Statistics. 426 pages. Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2013. An Introduction to Statistical Learning: with Applications in R. Springer Texts in Statistics. 426 pages."},{"key":"e_1_3_2_1_13_1","volume-title":"An analysis and survey of the development of mutation testing","author":"Jia Yue","year":"2010","unstructured":"Yue Jia and Mark Harman . 2010. An analysis and survey of the development of mutation testing . IEEE transactions on software engineering 37, 5 ( 2010 ), 649\u2013678. Yue Jia and Mark Harman. 2010. An analysis and survey of the development of mutation testing. IEEE transactions on software engineering 37, 5 (2010), 649\u2013678."},{"key":"e_1_3_2_1_14_1","volume-title":"MTUL: Towards Mutation Testing of Unsupervised Learning Systems","author":"Lu Yuteng","year":"2022","unstructured":"Yuteng Lu , Kaicheng Shao , Weidi Sun , and Meng Sun . 2022 . MTUL: Towards Mutation Testing of Unsupervised Learning Systems . In Dependable Software Engineering. Theories, Tools, and Applications, Wei Dong and Jean-Pierre Talpin (Eds.). Springer Nature Switzerland , Cham , 22\u201340. Yuteng Lu, Kaicheng Shao, Weidi Sun, and Meng Sun. 2022. MTUL: Towards Mutation Testing of Unsupervised Learning Systems. In Dependable Software Engineering. Theories, Tools, and Applications, Wei Dong and Jean-Pierre Talpin (Eds.). Springer Nature Switzerland, Cham, 22\u201340."},{"key":"e_1_3_2_1_15_1","volume-title":"DeepMutation: Mutation Testing of Deep Learning Systems. In 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE). 100\u2013111","author":"Ma Lei","year":"2018","unstructured":"Lei Ma , Fuyuan Zhang , Jiyuan Sun , Minhui Xue , Bo Li , Felix Juefei-Xu , Chao Xie , Li Li , Yang Liu , Jianjun Zhao , and Yadong Wang . 2018 . DeepMutation: Mutation Testing of Deep Learning Systems. In 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE). 100\u2013111 . Lei Ma, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Felix Juefei-Xu, Chao Xie, Li Li, Yang Liu, Jianjun Zhao, and Yadong Wang. 2018. DeepMutation: Mutation Testing of Deep Learning Systems. In 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE). 100\u2013111."},{"key":"e_1_3_2_1_16_1","volume-title":"M\u00fcller and Sarah Guido","author":"C.","year":"2016","unstructured":"A.\u00a0 C. M\u00fcller and Sarah Guido . 2016 . Introduction to Machine Learning with Python : A Guide for Data Scientists. O\u2019Reilly Media . 400 pages. A.\u00a0C. M\u00fcller and Sarah Guido. 2016. Introduction to Machine Learning with Python: A Guide for Data Scientists. O\u2019Reilly Media. 400 pages."},{"volume-title":"2021 IEEE\/ACM 43rd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER). 66\u201370","author":"Panichella Annibale","key":"e_1_3_2_1_17_1","unstructured":"Annibale Panichella and Cynthia C . \u00a0S. Liem. 2021. What Are We Really Testing in Mutation Testing for Machine Learning? A Critical Reflection . In 2021 IEEE\/ACM 43rd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER). 66\u201370 . Annibale Panichella and Cynthia C.\u00a0S. Liem. 2021. What Are We Really Testing in Mutation Testing for Machine Learning? A Critical Reflection. In 2021 IEEE\/ACM 43rd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER). 66\u201370."},{"key":"e_1_3_2_1_18_1","volume-title":"Proceedings of the 36th IEEE\/ACM International Conference on Automated Software Engineering(ASE \u201921)","author":"Riccio Vincenzo","year":"2022","unstructured":"Vincenzo Riccio , Nargiz Humbatova , Gunel Jahangirova , and Paolo Tonella . 2022 . DeepMetis: Augmenting a Deep Learning Test Set to Increase Its Mutation Score . In Proceedings of the 36th IEEE\/ACM International Conference on Automated Software Engineering(ASE \u201921) . IEEE Press, 355\u2013367. Vincenzo Riccio, Nargiz Humbatova, Gunel Jahangirova, and Paolo Tonella. 2022. DeepMetis: Augmenting a Deep Learning Test Set to Increase Its Mutation Score. In Proceedings of the 36th IEEE\/ACM International Conference on Automated Software Engineering(ASE \u201921). IEEE Press, 355\u2013367."},{"key":"e_1_3_2_1_19_1","volume-title":"Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing. 1\u20139.","author":"Santos Sebasti\u00e3o","year":"2021","unstructured":"Sebasti\u00e3o Santos , Beatriz Silveira , Vinicius Durelli , Rafael Durelli , Simone Souza , and Marcio Delamaro . 2021 . On Using Decision Tree Coverage Criteria for Testing Machine Learning Models . In Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing. 1\u20139. Sebasti\u00e3o Santos, Beatriz Silveira, Vinicius Durelli, Rafael Durelli, Simone Souza, and Marcio Delamaro. 2021. On Using Decision Tree Coverage Criteria for Testing Machine Learning Models. In Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software Testing. 1\u20139."},{"key":"e_1_3_2_1_20_1","volume-title":"MuNN: Mutation Analysis of Neural Networks. In 2018 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C). 108\u2013115","author":"Shen Weijun","year":"2018","unstructured":"Weijun Shen , Jun Wan , and Zhenyu Chen . 2018 . MuNN: Mutation Analysis of Neural Networks. In 2018 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C). 108\u2013115 . Weijun Shen, Jun Wan, and Zhenyu Chen. 2018. MuNN: Mutation Analysis of Neural Networks. In 2018 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C). 108\u2013115."},{"key":"e_1_3_2_1_21_1","first-page":"107","article-title":"A probabilistic framework for mutation testing in deep neural networks","volume":"155","author":"Tambon Florian","year":"2023","unstructured":"Florian Tambon , Foutse Khomh , and Giuliano Antoniol . 2023 . A probabilistic framework for mutation testing in deep neural networks . Information and Software Technology 155 (2023), 107 \u2013 129 . Florian Tambon, Foutse Khomh, and Giuliano Antoniol. 2023. A probabilistic framework for mutation testing in deep neural networks. Information and Software Technology 155 (2023), 107\u2013129.","journal-title":"Information and Software Technology"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"crossref","unstructured":"C. Wohlin P. Runeson M. H\u00f6st M.\u00a0C. Ohlsson B. Regnell and A. Wessl\u00e9n. 2012. Experimentation in Software Engineering. Springer. 236 pages.  C. Wohlin P. Runeson M. H\u00f6st M.\u00a0C. Ohlsson B. Regnell and A. Wessl\u00e9n. 2012. Experimentation in Software Engineering. Springer. 236 pages.","DOI":"10.1007\/978-3-642-29044-2"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"crossref","unstructured":"J.\u00a0M. Zhang M. Harman L. Ma and Y. Liu. 2020. Machine Learning Testing: Survey Landscapes and Horizons. IEEE Transactions on Software Engineering (Early Access) (2020) 1\u201337.  J.\u00a0M. Zhang M. Harman L. Ma and Y. Liu. 2020. Machine Learning Testing: Survey Landscapes and Horizons. IEEE Transactions on Software Engineering (Early Access) (2020) 1\u201337.","DOI":"10.1109\/TSE.2019.2962027"}],"event":{"name":"SAST 2023: 8th Brazilian Symposium on Systematic and Automated Software Testing","acronym":"SAST 2023","location":"Campo Grande, MS Brazil"},"container-title":["8th Brazilian Symposium on Systematic and Automated Software Testing"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3624032.3624038","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3624032.3624038","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:36:06Z","timestamp":1750178166000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3624032.3624038"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,25]]},"references-count":23,"alternative-id":["10.1145\/3624032.3624038","10.1145\/3624032"],"URL":"https:\/\/doi.org\/10.1145\/3624032.3624038","relation":{},"subject":[],"published":{"date-parts":[[2023,9,25]]},"assertion":[{"value":"2023-10-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}