{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,22]],"date-time":"2026-01-22T21:05:57Z","timestamp":1769115957028,"version":"3.49.0"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2023,9,1]],"date-time":"2023-09-01T00:00:00Z","timestamp":1693526400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,9,1]],"date-time":"2023-09-01T00:00:00Z","timestamp":1693526400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Kristiania University College"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Software Qual J"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>C# is one of the most widely used programming languages. However, to the best of our knowledge, there has been no work in the literature aimed at enabling search-based software testing techniques for applications running on the .NET platform, like the ones written in C#. In this paper, we propose a search-based approach and an open source tool to enable white-box testing for C# applications. The approach is integrated with a .NET bytecode instrumentation, in order to collect code coverage at runtime during the search. In addition, by taking advantage of <jats:italic>Branch Distance<\/jats:italic>, we define heuristics to better guide the search, e.g., how heuristically close it is to cover a branch in the source code. To empirically evaluate our technique, we integrated our tool into the <jats:sc>EvoMaster<\/jats:sc>\u00a0test generation tool and conducted experiments on three .NET RESTful APIs as case studies. Results show that our technique significantly outperforms gray-box testing tools in terms of code coverage.<\/jats:p>","DOI":"10.1007\/s11219-023-09645-1","type":"journal-article","created":{"date-parts":[[2023,9,1]],"date-time":"2023-09-01T16:02:56Z","timestamp":1693584176000},"page":"1439-1465","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":[".NET\/C# instrumentation for search-based software testing"],"prefix":"10.1007","volume":"31","author":[{"given":"Amid","family":"Golmohammadi","sequence":"first","affiliation":[]},{"given":"Man","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Andrea","family":"Arcuri","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,9,1]]},"reference":[{"issue":"3","key":"9645_CR1","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1002\/stvr.v16:3","volume":"16","author":"M Alshraideh","year":"2006","unstructured":"Alshraideh, M., & Bottaci, L. (2006). Search-based software test data generation for string data using program-specific search operators. Software Testing, Verification, and Reliability, 16(3), 175\u2013203. https:\/\/doi.org\/10.1002\/stvr.v16:3","journal-title":"Software Testing, Verification, and Reliability"},{"key":"9645_CR2","doi-asserted-by":"crossref","unstructured":"Arcuri, A.\u00a0(2018a). EvoMaster: Evolutionary multi-context automated system test generation. In\u00a0IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE.","DOI":"10.1109\/ICST.2018.00046"},{"key":"9645_CR3","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1016\/j.infsof.2018.05.003","volume":"104","author":"A Arcuri","year":"2018","unstructured":"Arcuri, A. (2018b). Test suite generation with the Many Independent Objective (MIO) algorithm. Information and Software Technology, 104, 195\u2013206.","journal-title":"Information and Software Technology"},{"issue":"4","key":"9645_CR4","doi-asserted-by":"publisher","first-page":"1959","DOI":"10.1007\/s10664-017-9570-9","volume":"23","author":"A Arcuri","year":"2018","unstructured":"Arcuri, A. (2018c). An experience report on applying software testing academic results in industry: We need usable automated test generation. Empirical Software Engineering, 23(4), 1959\u20131981.","journal-title":"Empirical Software Engineering"},{"issue":"1","key":"9645_CR5","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1145\/3293455","volume":"28","author":"A Arcuri","year":"2019","unstructured":"Arcuri, A. (2019). RESTful API automated test case generation with EvoMaster. ACM Transactions on Software Engineering and Methodology (TOSEM), 28(1), 3.","journal-title":"ACM Transactions on Software Engineering and Methodology (TOSEM)"},{"issue":"3","key":"9645_CR6","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1109\/MS.2020.3013820","volume":"38","author":"A Arcuri","year":"2020","unstructured":"Arcuri, A. (2020). Automated black-and white-box testing of restful APIs with EvoMaster. IEEE Software, 38(3), 72\u201378.","journal-title":"IEEE Software"},{"issue":"57","key":"9645_CR7","doi-asserted-by":"publisher","first-page":"2153","DOI":"10.21105\/joss.02153","volume":"6","author":"A Arcuri","year":"2021","unstructured":"Arcuri, A., Galeotti, J. P., Marculescu, B., & Zhang, M. (2021). EvoMaster: A search-based system test generation tool. Journal of Open Source Software, 6(57), 2153.","journal-title":"Journal of Open Source Software"},{"key":"9645_CR8","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.6651631","volume-title":"EMResearch\/EvoMaster: (v1.5.0)","author":"A Arcuri","year":"2022","unstructured":"Arcuri, A., ZhangMan, asmab89, Bogdan, Gol, A., Galeotti, J. P., Seran, A., L\u00f3pez, A. M., Aldasoro, A., Panichella, A., & Niemeyer, K. (2022). EMResearch\/EvoMaster: (v1.5.0). Zenodo. https:\/\/doi.org\/10.5281\/zenodo.6651631"},{"key":"9645_CR9","doi-asserted-by":"crossref","unstructured":"Arcuri, A., & Briand, L.\u00a0(2011). Adaptive random testing: An illusion of effectiveness? In ACM International Symposium on Software Testing and Analysis (ISSTA), pp. 265\u2013275.","DOI":"10.1145\/2001420.2001452"},{"issue":"3","key":"9645_CR10","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1002\/stvr.1486","volume":"24","author":"A Arcuri","year":"2014","unstructured":"Arcuri, A., & Briand, L. (2014). A hitchhiker\u2019s guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability (STVR), 24(3), 219\u2013250.","journal-title":"Software Testing, Verification and Reliability (STVR)"},{"issue":"4","key":"9645_CR11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3391533","volume":"29","author":"A Arcuri","year":"2020","unstructured":"Arcuri, A., & Galeotti, J. P. (2020). Handling SQL databases in automated system test generation. ACM Transactions on Software Engineering and Methodology (TOSEM), 29(4), 1\u201331.","journal-title":"ACM Transactions on Software Engineering and Methodology (TOSEM)"},{"issue":"1","key":"9645_CR12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3477271","volume":"31","author":"A Arcuri","year":"2021","unstructured":"Arcuri, A., & Galeotti, J. P. (2021). Enhancing search-based testing with testability transformations for existing APIs. ACM Transactions on Software Engineering and Methodology (TOSEM), 31(1), 1\u201334.","journal-title":"ACM Transactions on Software Engineering and Methodology (TOSEM)"},{"key":"9645_CR13","doi-asserted-by":"crossref","unstructured":"Atlidakis, V., Godefroid, P., & Polishchuk, M.\u00a0(2019). RESTler: Stateful REST API fuzzing. In ACM\/IEEE International Conference on Software Engineering (ICSE), pp. 748\u2013758.","DOI":"10.1109\/ICSE.2019.00083"},{"key":"9645_CR14","doi-asserted-by":"crossref","unstructured":"Belhadi, A., Zhang, M., & Arcuri, A.\u00a0(2022). Evolutionary-based automated testing for GraphQL APIs. In\u00a0Genetic and Evolutionary Computation Conference (GECCO).","DOI":"10.1145\/3520304.3528952"},{"key":"9645_CR15","unstructured":"ECMA-335. Common Language Infrastructure (CLI). https:\/\/www.ecma-international.org\/publications-and-standards\/standards\/ecma-335\/. Online, Accessed 3 August 2023"},{"key":"9645_CR16","unstructured":"EvoMaster. https:\/\/github.com\/EMResearch\/EvoMaster. Online, Accessed 3 August 2023"},{"key":"9645_CR17","unstructured":"EvoMaster Benchmark (EMB). https:\/\/github.com\/EMResearch\/EMB. Online, Accessed 20 May 2022."},{"key":"9645_CR18","doi-asserted-by":"crossref","unstructured":"Fraser, G., & Arcuri, A.\u00a0(2011). EvoSuite: Automatic test suite generation for object-oriented software. In ACM Symposium on the Foundations of Software Engineering (FSE), pp. 416\u2013419.","DOI":"10.1145\/2025113.2025179"},{"key":"9645_CR19","doi-asserted-by":"crossref","unstructured":"Garousi, V., Eskandar, M. M., & Herkilo\u011flu, K. (2016). Industry-academia collaborations in software testing: Experience and success stories from Canada and Turkey. Software Quality Journal, 1\u201353.","DOI":"10.1007\/s11219-016-9319-5"},{"issue":"5","key":"9645_CR20","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1109\/MS.2017.3641116","volume":"34","author":"V Garousi","year":"2017","unstructured":"Garousi, V., & Felderer, M. (2017). Worlds apart: A comparison of industry and academic focus areas in software testing. IEEE Software, 34(5), 38\u201345.","journal-title":"IEEE Software"},{"key":"9645_CR21","doi-asserted-by":"crossref","unstructured":"Garousi, V., Felderer, M., Kuhrmann, M., & Herkilo\u011flu, K.\u00a0(2017). What industry wants from academia in software testing? Hearing practitioners\u2019 opinions. In Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering, pp. 65\u201369. ACM.","DOI":"10.1145\/3084226.3084264"},{"issue":"4","key":"9645_CR22","doi-asserted-by":"publisher","first-page":"2540","DOI":"10.1007\/s10664-019-09711-y","volume":"24","author":"V Garousi","year":"2019","unstructured":"Garousi, V., Pfahl, D., Fernandes, J. M., Felderer, M., M\u00e4ntyl\u00e4, M. V., Shepherd, D., Arcuri, A., Co\u015fkun\u00e7ay, A., & Tekinerdogan, B. (2019). Characterizing industry-academia collaborations in software engineering: Evidence from 101 projects. Empirical Software Engineering, 24(4), 2540\u20132602.","journal-title":"Empirical Software Engineering"},{"issue":"1","key":"9645_CR23","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1145\/2379776.2379787","volume":"45","author":"M Harman","year":"2012","unstructured":"Harman, M., Mansouri, S. A., & Zhang, Y. (2012). Search-based software engineering: Trends, techniques and applications. ACM Computing Surveys (CSUR), 45(1), 11.","journal-title":"ACM Computing Surveys (CSUR)"},{"key":"9645_CR24","doi-asserted-by":"crossref","unstructured":"Hatfield-Dodds, Z., & Dygalo, D.\u00a0(2022). Deriving semantics-aware fuzzers from web API schemas. In 2022 IEEE\/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pp. 345\u2013346. IEEE","DOI":"10.1109\/ICSE-Companion55297.2022.9793781"},{"key":"9645_CR25","unstructured":"JetBrains Rider. https:\/\/www.jetbrains.com\/rider. Online, Accessed 3 August 2023"},{"key":"9645_CR26","doi-asserted-by":"publisher","unstructured":"Kim, M., Xin, Q., Sinha, S., & Orso, A.\u00a0(2022). Automated test generation for REST APIs: No time to rest yet. arXiv.\u00a0https:\/\/doi.org\/10.48550\/ARXIV.2204.08348,\u00a0https:\/\/arxiv.org\/abs\/2204.08348","DOI":"10.48550\/ARXIV.2204.08348"},{"issue":"8","key":"9645_CR27","doi-asserted-by":"publisher","first-page":"870","DOI":"10.1109\/32.57624","volume":"16","author":"B Korel","year":"1990","unstructured":"Korel, B. (1990). Automated software test data generation. IEEE Transactions on Software Engineering, 16(8), 870\u2013879.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"9645_CR28","doi-asserted-by":"publisher","first-page":"24738","DOI":"10.1109\/ACCESS.2021.3056505","volume":"9","author":"N Laranjeiro","year":"2021","unstructured":"Laranjeiro, N., Agnelo, J., & Bernardino, J. (2021). A black box tool for robustness testing of rest services. IEEE Access, 9, 24738\u201324754.","journal-title":"IEEE Access"},{"key":"9645_CR29","unstructured":"Lehre, P. K., & Yao, X.\u00a0(2007). Runtime analysis of (1+1) EA on computing unique input output sequences. In IEEE Congress on Evolutionary Computation (CEC), pp. 1882\u20131889."},{"key":"9645_CR30","doi-asserted-by":"crossref","unstructured":"Martin-Lopez, A., Segura, S., & Ruiz-Cort\u00e9s, A.\u00a0(2021). RESTest: Automated black-box testing of RESTful Web APIs. In ACM International Symposium on Software Testing and Analysis (ISSTA), pp. 682\u2013685. ACM.","DOI":"10.1145\/3460319.3469082"},{"key":"9645_CR31","doi-asserted-by":"crossref","unstructured":"Mao, K., Harman, M., & Jia, Y. (2016). Sapienz: Multi-objective automated testing for android applications. In ACM International Symposium on Software Testing and Analysis (ISSTA), pp. 94\u2013105. ACM.","DOI":"10.1145\/2931037.2931054"},{"key":"9645_CR32","doi-asserted-by":"crossref","unstructured":"Pacheco, C., Lahiri, S. K., Ernst, M. D., & Ball, T.\u00a0(2007). Feedback-directed random test generation. In ACM\/IEEE International Conference on Software Engineering (ICSE), pp. 75\u201384.","DOI":"10.1109\/ICSE.2007.37"},{"key":"9645_CR33","doi-asserted-by":"crossref","unstructured":"Panichella, S., Gambi, A., Zampetti, F., & Riccio, V.\u00a0(2021). SBST tool competition 2021. In 2021 IEEE\/ACM 14th International Workshop on Search-Based Software Testing (SBST), pp. 20\u201327. IEEE.","DOI":"10.1109\/SBST52555.2021.00011"},{"key":"9645_CR34","unstructured":"The State of the Octoverse. https:\/\/octoverse.github.com\/. Online, Accessed 3 August 2023"},{"key":"9645_CR35","doi-asserted-by":"crossref","unstructured":"Tillmann, N., &\u00a0de Halleux, J.\u00a0(2008). Pex \u2014 White box test generation for .NET. In TAP\u201908: International Conference on Tests And Proofs. LNCS, vol. 4966, pp. 134\u2013253. Springer.","DOI":"10.1007\/978-3-540-79124-9_10"},{"key":"9645_CR36","doi-asserted-by":"crossref","unstructured":"Viglianisi, E., Dallago, M., & Ceccato, M.\u00a0(2020). RESTTESTGEN: Automated black-box testing of restful APIs. In IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE.","DOI":"10.1109\/ICST46399.2020.00024"},{"key":"9645_CR37","doi-asserted-by":"crossref","unstructured":"Wu, H., Xu, L., Niu, X., & Nie, C.\u00a0(2022). Combinatorial testing of restful APIs. In ACM\/IEEE International Conference on Software Engineering (ICSE).","DOI":"10.1145\/3510003.3510151"},{"key":"9645_CR38","doi-asserted-by":"crossref","unstructured":"Zhang, M., & Arcuri, A. (2021). Adaptive hypermutation for search-based system test generation: A study on rest APIs with EvoMaster. ACM Transactions on Software Engineering and Methodology (TOSEM), 31(1).","DOI":"10.1145\/3464940"},{"key":"9645_CR39","unstructured":"Zhang, M., & Arcuri, A.\u00a0(2022). Open problems in fuzzing restful APIs: A comparison of tools. arXiv preprint: arXiv:2205.05325"},{"key":"9645_CR40","doi-asserted-by":"publisher","unstructured":"Zhang, M., Arcuri, A., Li, Y., Xue, K., Wang, Z., Huo, J., & Huang, W.\u00a0(2022). Fuzzing microservices in industry: Experience of applying EvoMaster at Meituan. arXiv. https:\/\/doi.org\/10.48550\/ARXIV.2208.03988, https:\/\/arxiv.org\/abs\/2208.03988","DOI":"10.48550\/ARXIV.2208.03988"},{"key":"9645_CR41","doi-asserted-by":"crossref","unstructured":"Zhang, M., Belhadi, A., & Arcuri, A.\u00a0(2022). Javascript instrumentation for search-based software testing: A study with restful APIs. In\u00a0IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE.","DOI":"10.1109\/ICST53961.2022.00022"},{"key":"9645_CR42","doi-asserted-by":"crossref","unstructured":"Zhang, M., Marculescu, B., & Arcuri, A.\u00a0(2019). Resource-based test case generation for restful web services. In Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1426\u20131434.","DOI":"10.1145\/3321707.3321815"},{"key":"9645_CR43","unstructured":".NET Platform. https:\/\/github.com\/dotnet. Online, Accessed 3 August 2023"}],"container-title":["Software Quality Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11219-023-09645-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11219-023-09645-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11219-023-09645-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,10]],"date-time":"2023-11-10T15:15:11Z","timestamp":1699629311000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11219-023-09645-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,1]]},"references-count":43,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["9645"],"URL":"https:\/\/doi.org\/10.1007\/s11219-023-09645-1","relation":{},"ISSN":["0963-9314","1573-1367"],"issn-type":[{"value":"0963-9314","type":"print"},{"value":"1573-1367","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,1]]},"assertion":[{"value":"6 July 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 September 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}