{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T16:28:53Z","timestamp":1783096133387,"version":"3.54.6"},"reference-count":81,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2020,7,6]],"date-time":"2020-07-06T00:00:00Z","timestamp":1593993600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001866","name":"Fonds National de la Recherche Luxembourg","doi-asserted-by":"publisher","award":["AFR-PhD-11289380-DroidMod, C15\/IS\/10404933\/COMMA"],"award-info":[{"award-number":["AFR-PhD-11289380-DroidMod, C15\/IS\/10404933\/COMMA"]}],"id":[{"id":"10.13039\/501100001866","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2020,10,31]]},"abstract":"<jats:p>Today, there are millions of third-party Android applications. Some of them are buggy or even malicious. To identify such applications, novel frameworks for automated black-box testing and dynamic analysis are being developed by the Android community. Code coverage is one of the most common metrics for evaluating effectiveness of these frameworks. Furthermore, code coverage is used as a fitness function for guiding evolutionary and fuzzy testing techniques. However, there are no reliable tools for measuring fine-grained code coverage in black-box Android app testing.<\/jats:p>\n          <jats:p>We present the Android Code coVerage Tool, ACVTool for short, that instruments Android apps and measures code coverage in the black-box setting at class, method and instruction granularity. ACVTool has successfully instrumented 96.9% of apps in our experiments. It introduces a negligible instrumentation time overhead, and its runtime overhead is acceptable for automated testing tools. We demonstrate practical value of ACVTool in a large-scale experiment with Sapienz, a state-of-the-art automated testing tool. Using ACVTool on the same cohort of apps, we have compared different coverage granularities applied by Sapienz in terms of the found amount of crashes. Our results show that none of the applied coverage granularities clearly outperforms others in this aspect.<\/jats:p>","DOI":"10.1145\/3395042","type":"journal-article","created":{"date-parts":[[2020,7,6]],"date-time":"2020-07-06T21:22:03Z","timestamp":1594070523000},"page":"1-35","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":25,"title":["Fine-grained Code Coverage Measurement in Automated Black-box Android Testing"],"prefix":"10.1145","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6789-6046","authenticated-orcid":false,"given":"Aleksandr","family":"Pilgun","sequence":"first","affiliation":[{"name":"SnT, University of Luxembourg, Luxembourg"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Olga","family":"Gadyatskaya","sequence":"additional","affiliation":[{"name":"LIACS, Leiden University, Leiden, Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yury","family":"Zhauniarovich","sequence":"additional","affiliation":[{"name":"Independent Researcher, Uzda, Belarus"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stanislav","family":"Dashevskyi","sequence":"additional","affiliation":[{"name":"Forescout Technologies, Eindhoven, Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Artsiom","family":"Kushniarou","sequence":"additional","affiliation":[{"name":"iTechArt Inc., Minsk, Belarus"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sjouke","family":"Mauw","sequence":"additional","affiliation":[{"name":"CSC 8 SnT, University of Luxembourg"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,7,6]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2017. dex2jar. Retrieved from https:\/\/github.com\/pxb1988\/dex2jar.  2017. dex2jar. Retrieved from https:\/\/github.com\/pxb1988\/dex2jar."},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the 2016 IEEE\/ACM 13th Working Conference on Mining Software Repositories (MSR\u201916)","author":"Allix K.","year":"2016"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-99241-9_1"},{"key":"e_1_2_1_4_1","volume-title":"Introduction to Software Testing","author":"Ammann Paul","edition":"2"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the International Conference on Software Engineering and Knowledge Engineering (SEKE\u201914)","author":"Arnatovich Yauhen","year":"2014"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/MOBILESoft.2017.2"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2509136.2509549"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSP.2017.43"},{"key":"e_1_2_1_9_1","unstructured":"Alexandre Bartel Jacques Klein Martin Monperrus Kevin Allix and Yves Le Traon. 2012. In-Vivo Bytecode Instrumentation for Improving Privacy on Android Smartphones in Uncertain Environments. arxiv:cs.CR\/1208.4536  Alexandre Bartel Jacques Klein Martin Monperrus Kevin Allix and Yves Le Traon. 2012. In-Vivo Bytecode Instrumentation for Improving Privacy on Android Smartphones in Uncertain Environments. arxiv:cs.CR\/1208.4536"},{"key":"e_1_2_1_10_1","volume-title":"Retrieved","author":"Gruver Ben","year":"2018"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197231.3197243"},{"key":"e_1_2_1_12_1","volume-title":"Retrieved","author":"Bornstein D.","year":"2008"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2017.35"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2017.31"},{"key":"e_1_2_1_15_1","volume-title":"CuriousDroid: Automated user interface interaction for Android application analysis sandboxes","author":"Carter Patrick"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.61"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2544173.2509552"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2015.89"},{"key":"e_1_2_1_19_1","volume-title":"Retrieved","author":"Cleron Mike","year":"2017"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-016-9443-7"},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the 2012 7th International Conference on Computing and Convergence Technology (ICCCT\u201912)","author":"Dai Shuaifu","year":"2012"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3278524"},{"key":"e_1_2_1_23_1","unstructured":"ELLA. 2016. A Tool for Binary Instrumentation of Android Apps. Retrieved from https:\/\/github.com\/saswatanand\/ella.  ELLA. 2016. A Tool for Binary Instrumentation of Android Apps. Retrieved from https:\/\/github.com\/saswatanand\/ella."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2660767"},{"key":"e_1_2_1_25_1","volume-title":"Retrieved Novermber 23","year":"2017"},{"key":"e_1_2_1_26_1","volume-title":"Retrieved","year":"2017"},{"key":"e_1_2_1_27_1","unstructured":"Google. 2018. Dalvik bytecode. Retrieved from https:\/\/source.android.com\/devices\/tech\/dalvik\/dalvik-bytecode.  Google. 2018. Dalvik bytecode. Retrieved from https:\/\/source.android.com\/devices\/tech\/dalvik\/dalvik-bytecode."},{"key":"e_1_2_1_28_1","volume-title":"smali. Retrieved December, 2","year":"2018"},{"key":"e_1_2_1_29_1","unstructured":"Google. 2018. Test your app. Retrieved from https:\/\/developer.android.com\/studio\/test\/index.html.  Google. 2018. Test your app. Retrieved from https:\/\/developer.android.com\/studio\/test\/index.html."},{"key":"e_1_2_1_30_1","volume-title":"Retrieved","year":"2018"},{"key":"e_1_2_1_31_1","volume-title":"Retrieved","author":"Inc. Google.","year":"2019"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2017.72"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.14232\/actacyb.21.3.2014.10"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.14722\/bar.2018.23014"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/MobServ.2015.38"},{"key":"e_1_2_1_36_1","volume-title":"Proceedings of the IEEE International Conference on Progress in Informatics and Computing (PIC\u201915)","author":"Huang Shang-Yi","year":"2015"},{"key":"e_1_2_1_37_1","unstructured":"JaCoCo. 2018. Java Code Coverage Library. Retrieved from http:\/\/www.jacoco.org\/.  JaCoCo. 2018. Java Code Coverage Library. Retrieved from http:\/\/www.jacoco.org\/."},{"key":"e_1_2_1_38_1","first-page":"1","article-title":"Automated testing of android apps: A systematic literature review","volume":"68","author":"Kong P.","year":"2018","journal-title":"IEEE Trans. Reliabil."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2018.00020"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00021"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2013.6698891"},{"key":"e_1_2_1_42_1","unstructured":"Y. Li. 2016. AndroCov. Measure test coverage without source code. Retrieved from https:\/\/github.com\/ylimit\/androcov.  Y. Li. 2016. AndroCov. Measure test coverage without source code. Retrieved from https:\/\/github.com\/ylimit\/androcov."},{"key":"e_1_2_1_43_1","volume-title":"Proceedings of the 2017 IEEE\/ACM 39th International Conference on Software Engineering Companion (ICSE-C\u201917)","author":"Li Yuanchun","year":"2017"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106275"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2017.27"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2017.7884662"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASE.2016.27"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301293.3302363"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491411.2491450"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.5555\/2663608.2663613"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931054"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-C.2017.16"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3194733.3194742"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3278484"},{"key":"e_1_2_1_55_1","volume-title":"Retrieved","author":"Technologies Qualcomm","year":"2019"},{"key":"e_1_2_1_56_1","volume-title":"EMMA: Java Code Coverage Tool.","author":"Rubtsov V.","year":"2006"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106250"},{"key":"e_1_2_1_58_1","volume-title":"Retrieved","author":"Software Scooter","year":"2019"},{"key":"e_1_2_1_59_1","unstructured":"PassMark Software. 2018. Passmark. Interpreting your Results from PerformanceTest. Retrieved from https:\/\/www.passmark.com\/support\/performancetest\/interpreting_test_results.htm.  PassMark Software. 2018. Passmark. Interpreting your Results from PerformanceTest. Retrieved from https:\/\/www.passmark.com\/support\/performancetest\/interpreting_test_results.htm."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.5555\/3155562.3155570"},{"key":"e_1_2_1_61_1","unstructured":"JetBrains s.r.o. 2017. Code Coverage. Retrieved from https:\/\/www.jetbrains.com\/help\/idea\/2017.1\/code-coverage.html.  JetBrains s.r.o. 2017. Code Coverage. Retrieved from https:\/\/www.jetbrains.com\/help\/idea\/2017.1\/code-coverage.html."},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106298"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2015.23145"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2016.61"},{"key":"e_1_2_1_65_1","volume-title":"Hendren","author":"Vallee-Rai Raja","year":"1998"},{"key":"e_1_2_1_66_1","first-page":"101","article-title":"A critique and improvement of the CL common language effect size statistics of McGraw and Wong","volume":"25","author":"Vargha Andr\u00e1s","year":"2000","journal-title":"J. Educ. Behav. Stat."},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3240465"},{"key":"e_1_2_1_68_1","unstructured":"R. Wi\u015bniewski and C. Tumbleson. 2017. Apktool - A tool for reverse engineering 3rd party closed binary Android apps. Retrieved from https:\/\/ibotpeaches.github.io\/Apktool\/.  R. Wi\u015bniewski and C. Tumbleson. 2017. Apktool - A tool for reverse engineering 3rd party closed binary Android apps. Retrieved from https:\/\/ibotpeaches.github.io\/Apktool\/."},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.5555\/2349018"},{"key":"e_1_2_1_70_1","volume-title":"Proceedings of the Network and Distributed System Security Symposium (NDSS\u201916)","volume":"16","author":"Michelle"},{"key":"e_1_2_1_71_1","unstructured":"Jiwei Yan Tianyong Wu Jun Yan and Jian Zhang. 2016. Target Directed Event Sequence Generation for Android Applications. arxiv:cs.SE\/1607.03258  Jiwei Yan Tianyong Wu Jun Yan and Jian Zhang. 2016. Target Directed Event Sequence Generation for Android Applications. arxiv:cs.SE\/1607.03258"},{"key":"e_1_2_1_72_1","volume-title":"APK Instrumentation Library. Retrieved","author":"Yang K.","year":"2018"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1093\/comjnl\/bxm021"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2015.125"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1002\/stv.430"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2016.03.065"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2983958"},{"key":"e_1_2_1_78_1","volume-title":"Proceedings of the 2019 IEEE\/ACM 41st International Conference on Software Engineering (ICSE\u201919)","author":"Zhao Yu"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/2699026.2699105"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-43936-4_9"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1109\/ARES.2015.70"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3395042","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3395042","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:02:47Z","timestamp":1750197767000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3395042"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,6]]},"references-count":81,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,10,31]]}},"alternative-id":["10.1145\/3395042"],"URL":"https:\/\/doi.org\/10.1145\/3395042","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,7,6]]},"assertion":[{"value":"2018-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-07-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}