{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T07:47:00Z","timestamp":1775807220718,"version":"3.50.1"},"reference-count":31,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2022,4,14]],"date-time":"2022-04-14T00:00:00Z","timestamp":1649894400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Software and symmetric testing methodologies are primarily used in detecting software defects, but these testing methodologies need to be optimized to mitigate the wasting of resources. As mobile applications are becoming more prevalent in recent times, the need to have mobile applications that satisfy software quality through testing cannot be overemphasized. Testing suites and software quality assurance techniques have also become prevalent, which underscores the need to evaluate the efficacy of these tools in the testing of the applications. Mutation testing is one such technique, which is the process of injecting small changes into the software under test (SUT), thereby creating mutants. These mutants are then tested using mutation testing techniques alongside the SUT to determine the effectiveness of test suites through mutation scoring. Although mutation testing is effective, the cost of implementing it, due to the problem of equivalent mutants, is very high. Many research works gave varying solutions to this problem, but none used a standardized dataset. In this research work, we employed a standard mutant dataset tool called MutantBench to generate our data. Subsequently, an Abstract Syntax Tree (AST) was used in conjunction with a tree-based convolutional neural network (TBCNN) as our deep learning model to automate the classification of the equivalent mutants to reduce the cost of mutation testing in software testing of android applications. The result shows that the proposed model produces a good accuracy rate of 94%, as well as other performance metrics such as recall (96%), precision (89%), F1-score (92%), and Matthew\u2019s correlation coefficients (88%) with fewer False Negatives and False Positives during testing, which is significant as it implies that there is a decrease in the risk of misclassification.<\/jats:p>","DOI":"10.3390\/sym14040820","type":"journal-article","created":{"date-parts":[[2022,4,20]],"date-time":"2022-04-20T00:22:43Z","timestamp":1650414163000},"page":"820","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Automatic Classification of Equivalent Mutants in Mutation Testing of Android Applications"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0901-2326","authenticated-orcid":false,"given":"Muhammad Bello","family":"Kusharki","sequence":"first","affiliation":[{"name":"Department of Information and Communication Technology, National Defence College, Abuja 900128, Nigeria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sanjay","family":"Misra","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Communication, \u00d8stfold University College, P.O. Box 700, NO-1757 Halden, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4020-7175","authenticated-orcid":false,"given":"Bilkisu","family":"Muhammad-Bello","sequence":"additional","affiliation":[{"name":"Software Engineering & Information Technology Department, Nile University of Nigeria, Abuja 900001, Nigeria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7015-8741","authenticated-orcid":false,"given":"Ibrahim Anka","family":"Salihu","sequence":"additional","affiliation":[{"name":"Software Engineering & Information Technology Department, Nile University of Nigeria, Abuja 900001, Nigeria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bharti","family":"Suri","sequence":"additional","affiliation":[{"name":"University School of Information and Communication Technology, GGS Indraprastha University, Delhi 110078, India"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,4,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.infsof.2016.04.012","article-title":"Mutation operators for testing Android apps","volume":"81","author":"Deng","year":"2017","journal-title":"Inf. Softw. Technol."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Strug, J., and Strug, B. (2012). LNCS 7641\u2014Machine Learning Approach in Mutation Testing. IFIP International Conference on Testing Software and Systems, Springer.","DOI":"10.1007\/978-3-642-34691-0_15"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e1728","DOI":"10.1002\/stvr.1728","article-title":"Performance Mutation Testing","volume":"31","author":"Segura","year":"2021","journal-title":"Softw. Test. Verif. Reliab."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"17158","DOI":"10.1109\/ACCESS.2019.2895504","article-title":"AMOGA: A Static-Dynamic Model Generation Strategy for Mobile Apps Testing","volume":"7","author":"Salihu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"e2238","DOI":"10.1002\/smr.2238","article-title":"A machine learning approach for classification of equivalent mutants","volume":"32","author":"Naeem","year":"2020","journal-title":"J. Softw. Evol. Process"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"116309","DOI":"10.1109\/ACCESS.2019.2936948","article-title":"Deep Learning with Customized Abstract Syntax Tree for Bug Localization","volume":"7","author":"Liang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Peacock, S., Deng, L., Dehlinger, J., and Chakraborty, S. (2021, January 12\u201316). Automatic Equivalent Mutants Classification Using Abstract Syntax Tree Neural Networks. Proceedings of the 2021 IEEE 14th International Conference on Software Testing, Verification and Validation Workshops (ICSTW), Porto de Galinhas, Brazil.","DOI":"10.1109\/ICSTW52544.2021.00016"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Escobar-Vel\u00e1squez, C., Riveros, D., and Linares-V\u00e1squez, M. (2020, January 8\u201313). MutAPK 2.0: A tool for reducing mutation testing effort of Android apps. Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Virtual Event.","DOI":"10.1145\/3368089.3417942"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"570","DOI":"10.1109\/TSE.2012.39","article-title":"Validating Second-Order Mutation at System Level","volume":"39","author":"Mateo","year":"2012","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Van Hijfte, L., and Oprescu, A. (2021, January 12\u201316). MutantBench: An Equivalent Mutant Problem Comparison Framework. Proceedings of the 2021 IEEE 14th International Conference on Software Testing, Verification and Validation Workshops (ICSTW), Porto de Galinhas, Brazil.","DOI":"10.1109\/ICSTW52544.2021.00015"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"106317","DOI":"10.1016\/j.infsof.2020.106317","article-title":"An experimental and practical study on the equivalent mutant connection: An evolutionary approach","volume":"124","author":"Chicano","year":"2020","journal-title":"Inf. Softw. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1109\/TSE.2017.2684805","article-title":"Detecting Trivial Mutant Equivalences via Compiler Optimisations","volume":"44","author":"Kintis","year":"2017","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ma, L., Zhang, F., Sun, J., Xue, M., Li, B., Juefei-Xu, F., Xie, C., Li, L., Liu, Y., and Zhao, J. (2018, January 15\u201318). DeepMutation: Mutation Testing of Deep Learning Systems. Proceedings of the 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE), Memphis, TN, USA.","DOI":"10.1109\/ISSRE.2018.00021"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Linares-V\u00e1squez, M., Bavota, G., Tufano, M., Moran, K., Di Penta, M., Vendome, C., Bernal-C\u00e1rdenas, C., and Poshyvanyk, D. (2017, January 4\u20138). Enabling mutation testing for Android apps. Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, Association for Computing Machinery (ACM), Paderborn, Germany.","DOI":"10.1145\/3106237.3106275"},{"key":"ref_15","unstructured":"Moran, K., Tufano, M., Bernal-C\u00e1rdenas, C., Linares-V\u00e1squez, M., Bavota, G., Vendome, C., Di Penta, M., and Poshyvanyk, D. (June, January 27). MDroid+: A mutation testing framework for android. Proceedings of the 40th International Conference on Software Engineering: Companion, Gothenburg, Sweden."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Da Silva, H.N., Farah, P.R., Mendon\u00e7a, W.D.F., and Vergilio, S.R. (2019, January 23\u201327). Assessing Android Test Data Generation Tools via Mutation Testing. Proceedings of the IV Brazilian Symposium on Systematic and Automated Software Testing\u2014SAST 2019, Salvador, Brazil.","DOI":"10.1145\/3356317.3356320"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Yao, X., Harman, M., and Jia, Y. (2014\u20137, January 31). A study of equivalent and stubborn mutation operators using human analysis of equivalence. Proceedings of the 36th International Conference on Software Engineering, Hyderabad, India.","DOI":"10.1145\/2568225.2568265"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.datak.2017.07.003","article-title":"Automatically classifying source code using tree-based approaches","volume":"114","author":"Phan","year":"2018","journal-title":"Data Knowl. Eng."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Hu, Q., Ma, L., Xie, X., Yu, B., Liu, Y., and Zhao, J. (2019, January 11\u201315). DeepMutation++: A Mutation Testing Framework for Deep Learning Systems. Proceedings of the 2019 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE), San Diego, CA, USA.","DOI":"10.1109\/ASE.2019.00126"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Baer, M., Oster, N., and Philippsen, M. (2020, January 24\u201328). MutantDistiller: Using Symbolic Execution for Automatic Detection of Equivalent Mutants and Generation of Mutant Killing Tests. Proceedings of the 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), Porto, Portugal.","DOI":"10.1109\/ICSTW50294.2020.00055"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Tang, D., Qin, B., and Liu, T. (2015, January 17\u201321). Document Modeling with Gated Recurrent Neural Network for Sentiment Classification. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.","DOI":"10.18653\/v1\/D15-1167"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.neunet.2019.09.010","article-title":"Tree-CNN: A hierarchical Deep Convolutional Neural Network for incremental learning","volume":"121","author":"Roy","year":"2020","journal-title":"Neural Netw."},{"key":"ref_23","unstructured":"Mou, L., Li, G., Zhang, L., Wang, T., and Jin, Z. (2022, March 21). Convolutional Neural Networks over Tree Structures for Programming Language Processing. Available online: http:\/\/arxiv.org\/abs\/1409.5718."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"23","DOI":"10.4018\/IJOSSP.2020010102","article-title":"Mutation Testing to Evaluate Android Applications","volume":"11","author":"Saifan","year":"2020","journal-title":"Int. J. Open Source Softw. Process."},{"key":"ref_25","unstructured":"Mikolov, T., Sutskever, I., Chen, K., Corrado, G., and Dean, J. (2022, March 21). Distributed Representations of Words and Phrases and Their Compositionality. Available online: http:\/\/arxiv.org\/abs\/1310.4546."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"180","DOI":"10.24138\/jcomss.v16i2.1032","article-title":"A Speech Quality Classifier based on Tree-CNN Algorithm that Considers Network Degradations","volume":"16","author":"Vieira","year":"2020","journal-title":"J. Commun. Softw. Syst."},{"key":"ref_27","unstructured":"Bui, N.D.Q., Jiang, L., and Yu, Y. (2022, March 21). Cross-Language Learning for Program Classification Using Bilateral Tree-Based Convolutional Neural Networks. Available online: http:\/\/arxiv.org\/abs\/1710.06159."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"170844","DOI":"10.1109\/ACCESS.2019.2953696","article-title":"An Abstract Syntax Tree Encoding Method for Cross-Project Defect Prediction","volume":"7","author":"Cai","year":"2019","journal-title":"IEEE Access"},{"key":"ref_29","unstructured":"Kingma, D.P., and Ba, J. (2022, March 21). Adam: A Method for Stochastic Optimization. Available online: https:\/\/arxiv.org\/abs\/1412.6980."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Chicco, D., and Jurman, G. (2020). The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genom., 21.","DOI":"10.1186\/s12864-019-6413-7"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13040-021-00244-z","article-title":"The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation","volume":"14","author":"Chicco","year":"2021","journal-title":"BioData Min."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/14\/4\/820\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:54:08Z","timestamp":1760136848000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/14\/4\/820"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,14]]},"references-count":31,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,4]]}},"alternative-id":["sym14040820"],"URL":"https:\/\/doi.org\/10.3390\/sym14040820","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,14]]}}}