{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T14:29:05Z","timestamp":1764944945923,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":29,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,9,25]],"date-time":"2023-09-25T00:00:00Z","timestamp":1695600000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,9,25]]},"DOI":"10.1145\/3624032.3624034","type":"proceedings-article","created":{"date-parts":[[2023,10,17]],"date-time":"2023-10-17T18:15:18Z","timestamp":1697566518000},"page":"5-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["An Experimental Study Evaluating Cost, Adequacy, and Effectiveness of Pynguin's Test Sets"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-6071-1418","authenticated-orcid":false,"given":"Lucca","family":"Guerino","sequence":"first","affiliation":[{"name":"Departamento de Computa\u00e7\u00e3o, Universidade Federal de S\u00e3o Carlos, Brazil"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5902-1672","authenticated-orcid":false,"given":"Auri","family":"Vincenzi","sequence":"additional","affiliation":[{"name":"Departamento de Computa\u00e7\u00e3o, Universidade Federal de S\u00e3o Carlos, Brazil"}]}],"member":"320","published-online":{"date-parts":[[2023,10,17]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"21st Belgium-Netherlands Software Evolution Workshop \u2013 BENEVOL\u20192022(CEUR Workshop Proceedings, Vol.\u00a01). Mons, 10","author":"Abdi Mehrdad","year":"2022","unstructured":"Mehrdad Abdi and Serge Demeyer . 2022 . Steps towards zero-touch mutation testing in Pharo . In 21st Belgium-Netherlands Software Evolution Workshop \u2013 BENEVOL\u20192022(CEUR Workshop Proceedings, Vol.\u00a01). Mons, 10 . Mehrdad Abdi and Serge Demeyer. 2022. Steps towards zero-touch mutation testing in Pharo. In 21st Belgium-Netherlands Software Evolution Workshop \u2013 BENEVOL\u20192022(CEUR Workshop Proceedings, Vol.\u00a01). Mons, 10."},{"key":"e_1_3_2_1_2_1","volume-title":"How Do Java Mutation Tools Differ?Commun. ACM 65, 12 (Nov","author":"Amalfitano Domenico","year":"2022","unstructured":"Domenico Amalfitano , Ana C.\u00a0R. Paiva , Alexis Inquel , Lu\u00eds Pinto , Anna\u00a0Rita Fasolino , and Ren\u00e9 Just . 2022. How Do Java Mutation Tools Differ?Commun. ACM 65, 12 (Nov . 2022 ), 74\u201389. https:\/\/doi.org\/10.1145\/3526099 Place : New York, NY, USA Publisher : Association for Computing Machinery. 10.1145\/3526099 Domenico Amalfitano, Ana C.\u00a0R. Paiva, Alexis Inquel, Lu\u00eds Pinto, Anna\u00a0Rita Fasolino, and Ren\u00e9 Just. 2022. How Do Java Mutation Tools Differ?Commun. ACM 65, 12 (Nov. 2022), 74\u201389. https:\/\/doi.org\/10.1145\/3526099 Place: New York, NY, USA Publisher: Association for Computing Machinery."},{"key":"#cr-split#-e_1_3_2_1_3_1.1","doi-asserted-by":"crossref","unstructured":"Filipe\u00a0Santos Araujo and Auri Vincenzi. 2020. How far are we from testing a program in a completely automated way considering the mutation testing criterion at unit level?. In Anais do Simp\u00f3sio Brasileiro de Qualidade de Software (SBQS). SBC 151-159. https:\/\/doi.org\/10.1145\/3439961.3439977 10.1145\/3439961.3439977","DOI":"10.1145\/3439961.3439977"},{"key":"#cr-split#-e_1_3_2_1_3_1.2","doi-asserted-by":"crossref","unstructured":"Filipe\u00a0Santos Araujo and Auri Vincenzi. 2020. How far are we from testing a program in a completely automated way considering the mutation testing criterion at unit level?. In Anais do Simp\u00f3sio Brasileiro de Qualidade de Software (SBQS). SBC 151-159. https:\/\/doi.org\/10.1145\/3439961.3439977","DOI":"10.1145\/3439961.3439977"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2018.05.003"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2014.2372785"},{"key":"e_1_3_2_1_6_1","volume-title":"Cosmic Ray: mutation testing for Python. (Nov","author":"Bingham Austin","year":"2021","unstructured":"Austin Bingham . 2021. Cosmic Ray: mutation testing for Python. (Nov . 2021 ). https:\/\/github.com\/sixty-north\/cosmic-ray Austin Bingham. 2021. Cosmic Ray: mutation testing for Python. (Nov. 2021). https:\/\/github.com\/sixty-north\/cosmic-ray"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2166929.2166938"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3555228.3555233"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897010.2897020"},{"key":"e_1_3_2_1_10_1","volume-title":"MutPy: a mutation testing tool for Python 3.x source code. (Nov","author":"Hossner Philipp","year":"2021","unstructured":"Philipp Hossner , Konrad Ha\u0142as , Steven Myint , and Andreas Mueller . 2021. MutPy: a mutation testing tool for Python 3.x source code. (Nov . 2021 ). https:\/\/github.com\/mutpy\/mutpy Philipp Hossner, Konrad Ha\u0142as, Steven Myint, and Andreas Mueller. 2021. MutPy: a mutation testing tool for Python 3.x source code. (Nov. 2021). https:\/\/github.com\/mutpy\/mutpy"},{"key":"e_1_3_2_1_11_1","volume-title":"Mutmut: a Python mutation testing system. (Nov","author":"Hovm\u00f6ller Anders","year":"2021","unstructured":"Anders Hovm\u00f6ller . 2021. Mutmut: a Python mutation testing system. (Nov . 2021 ). https:\/\/hackernoon.com\/mutmut-a-python-mutation-testing-system-9b9639356c78 Anders Hovm\u00f6ller. 2021. Mutmut: a Python mutation testing system. (Nov. 2021). https:\/\/hackernoon.com\/mutmut-a-python-mutation-testing-system-9b9639356c78"},{"key":"e_1_3_2_1_12_1","volume-title":"Mutatest: Python mutation testing. (Nov.","author":"Kepner Evan","year":"2011","unstructured":"Evan Kepner . 2011 . Mutatest: Python mutation testing. (Nov. 2011). https:\/\/github.com\/EvanKepner\/mutatest Evan Kepner. 2011. Mutatest: Python mutation testing. (Nov. 2011). https:\/\/github.com\/EvanKepner\/mutatest"},{"key":"e_1_3_2_1_13_1","first-page":"2643","volume-title":"2019 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 1226\u20131229","author":"Lukasczyk Stephan","year":"2019","unstructured":"Stephan Lukasczyk . 2019 . Generating Tests to Analyse Dynamically-Typed Programs . In 2019 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 1226\u20131229 . https:\/\/doi.org\/10.1109\/ASE.2019.00146 ISSN: 2643 - 1572 . 10.1109\/ASE.2019.00146 Stephan Lukasczyk. 2019. Generating Tests to Analyse Dynamically-Typed Programs. In 2019 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 1226\u20131229. https:\/\/doi.org\/10.1109\/ASE.2019.00146 ISSN: 2643-1572."},{"key":"e_1_3_2_1_14_1","first-page":"2","article-title":"An empirical study of automated unit test generation for Python","volume":"28","author":"Lukasczyk Stephan","year":"2023","unstructured":"Stephan Lukasczyk , Florian Kroi\u00df , and Gordon Fraser . 2023 . An empirical study of automated unit test generation for Python . Empirical Software Engineering 28 , 2 (Jan. 2023), 36. https:\/\/doi.org\/10.1007\/s10664-022-10248-w 10.1007\/s10664-022-10248-w Stephan Lukasczyk, Florian Kroi\u00df, and Gordon Fraser. 2023. An empirical study of automated unit test generation for Python. Empirical Software Engineering 28, 2 (Jan. 2023), 36. https:\/\/doi.org\/10.1007\/s10664-022-10248-w","journal-title":"Empirical Software Engineering"},{"volume-title":"Code Complete","author":"McConnell Steve","key":"e_1_3_2_1_15_1","unstructured":"Steve McConnell . 2004. Code Complete , Second Edition. Microsoft Press , USA. Steve McConnell. 2004. Code Complete, Second Edition. Microsoft Press, USA."},{"key":"#cr-split#-e_1_3_2_1_16_1.1","doi-asserted-by":"crossref","unstructured":"Carlos Pacheco and Michael\u00a0D. Ernst. 2007. Randoop: Feedback-directed Random Testing for Java. In Companion to the 22Nd ACM SIGPLAN Conference on Object-oriented Programming Systems and Applications Companion(OOPSLA '07). ACM Montreal Quebec Canada 815-816. https:\/\/doi.org\/10.1145\/1297846.1297902 10.1145\/1297846.1297902","DOI":"10.1145\/1297846.1297902"},{"key":"#cr-split#-e_1_3_2_1_16_1.2","doi-asserted-by":"crossref","unstructured":"Carlos Pacheco and Michael\u00a0D. Ernst. 2007. Randoop: Feedback-directed Random Testing for Java. In Companion to the 22Nd ACM SIGPLAN Conference on Object-oriented Programming Systems and Applications Companion(OOPSLA '07). ACM Montreal Quebec Canada 815-816. https:\/\/doi.org\/10.1145\/1297846.1297902","DOI":"10.1145\/1297846.1297902"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2007.37"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2015.7102604"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2663435"},{"key":"e_1_3_2_1_20_1","volume-title":"A Detailed Investigation of the Effectiveness of Whole Test Suite Generation. Empirical Softw. Engg. 22, 2 (April","author":"Rojas Jos\u00e9\u00a0Miguel","year":"2017","unstructured":"Jos\u00e9\u00a0Miguel Rojas , Mattia Vivanti , Andrea Arcuri , and Gordon Fraser . 2017. A Detailed Investigation of the Effectiveness of Whole Test Suite Generation. Empirical Softw. Engg. 22, 2 (April 2017 ), 852\u2013893. https:\/\/doi.org\/10.1007\/s10664-015-9424-2 Place : USA Publisher: Kluwer Academic Publishers . 10.1007\/s10664-015-9424-2 Jos\u00e9\u00a0Miguel Rojas, Mattia Vivanti, Andrea Arcuri, and Gordon Fraser. 2017. A Detailed Investigation of the Effectiveness of Whole Test Suite Generation. Empirical Softw. Engg. 22, 2 (April 2017), 852\u2013893. https:\/\/doi.org\/10.1007\/s10664-015-9424-2 Place: USA Publisher: Kluwer Academic Publishers."},{"key":"#cr-split#-e_1_3_2_1_21_1.1","doi-asserted-by":"crossref","unstructured":"Abdelilah Sakti Gilles Pesant and Yann-Ga\u00ebl Gu\u00e9h\u00e9neuc. 2015. JTExpert at the Third Unit Testing Tool Competition. 52-55. https:\/\/doi.org\/10.1109\/SBST.2015.20 10.1109\/SBST.2015.20","DOI":"10.1109\/SBST.2015.20"},{"key":"#cr-split#-e_1_3_2_1_21_1.2","doi-asserted-by":"crossref","unstructured":"Abdelilah Sakti Gilles Pesant and Yann-Ga\u00ebl Gu\u00e9h\u00e9neuc. 2015. JTExpert at the Third Unit Testing Tool Competition. 52-55. https:\/\/doi.org\/10.1109\/SBST.2015.20","DOI":"10.1109\/SBST.2015.20"},{"key":"e_1_3_2_1_22_1","first-page":"e2490","article-title":"AmPyfier: Test amplification in Python","volume":"34","author":"Schoofs Ebert","year":"2022","unstructured":"Ebert Schoofs , Mehrdad Abdi , and Serge Demeyer . 2022 . AmPyfier: Test amplification in Python . Journal of Software: Evolution and Process 34 , 11 (2022), e2490 . https:\/\/doi.org\/10.1002\/smr.2490 10.1002\/smr.2490 Ebert Schoofs, Mehrdad Abdi, and Serge Demeyer. 2022. AmPyfier: Test amplification in Python. Journal of Software: Evolution and Process 34, 11 (2022), e2490. https:\/\/doi.org\/10.1002\/smr.2490","journal-title":"Journal of Software: Evolution and Process"},{"key":"e_1_3_2_1_23_1","unstructured":"TIOBE Software BV. 2023. TIOBE Index. https:\/\/www.tiobe.com\/tiobe-index\/  TIOBE Software BV. 2023. TIOBE Index. https:\/\/www.tiobe.com\/tiobe-index\/"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2994291.2994295"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/SBST52555.2021.00012"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1986036"}],"event":{"name":"SAST 2023: 8th Brazilian Symposium on Systematic and Automated Software Testing","acronym":"SAST 2023","location":"Campo Grande, MS Brazil"},"container-title":["8th Brazilian Symposium on Systematic and Automated Software Testing"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3624032.3624034","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3624032.3624034","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:36:06Z","timestamp":1750178166000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3624032.3624034"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,25]]},"references-count":29,"alternative-id":["10.1145\/3624032.3624034","10.1145\/3624032"],"URL":"https:\/\/doi.org\/10.1145\/3624032.3624034","relation":{},"subject":[],"published":{"date-parts":[[2023,9,25]]},"assertion":[{"value":"2023-10-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}