{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T15:52:02Z","timestamp":1784303522098,"version":"3.55.0"},"reference-count":63,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2025,9,22]],"date-time":"2025-09-22T00:00:00Z","timestamp":1758499200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,9,22]],"date-time":"2025-09-22T00:00:00Z","timestamp":1758499200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001794","name":"The University of Queensland","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001794","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2025,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Android dominates the mobile operating system market, yet ensuring the quality and reliability of Android applications remains a persistent challenge. The diversity of devices, screen sizes, and OS versions complicates testing, leading to fragmented adoption of best practices. Despite advancements in automated testing, there is Limited empirical evidence on how developers test Android applications and the extent to which existing tools and frameworks are utilized effectively. In this paper, we aim to investigate the current state of Android app testing, identifying key challenges, Limitations, and best practices. Specifically, we assess the adoption of automated testing, test coverage levels, and the impact of testing practices on software quality. We conduct an experimental study on 2965 open-source Android apps, examining the quantity and coverage of the tests used for open-source Android app development. We further conduct a survey to gather more insights in testing practices from Android app developers and testers. The results reveal a limited adoption of testing among Android app developers, a restricted range of testing tools and frameworks being used, and low code and API coverage in testing. This investigation shows that current Android app testing practices are lacking the use of automated testing tools and embarks on a need for more awareness and adoption of state-of-the-art testing tools and techniques.<\/jats:p>","DOI":"10.1007\/s10664-025-10726-x","type":"journal-article","created":{"date-parts":[[2025,9,22]],"date-time":"2025-09-22T12:36:13Z","timestamp":1758544573000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Why android app testing falls short: empirical insights from open-source projects and a practitioner survey"],"prefix":"10.1007","volume":"30","author":[{"given":"Tarek","family":"Mahmud","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Meiru","family":"Che","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anne","family":"Ngu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1404-4560","authenticated-orcid":false,"given":"Guowei","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,9,22]]},"reference":[{"key":"10726_CR1","unstructured":"Android vs. apple market share: Leading mobile operating systems (os) (2023). https:\/\/bankmycell.com\/blog\/android-vs-apple-market-share\/"},{"key":"10726_CR2","doi-asserted-by":"crossref","unstructured":"Arif KS, Ali U (2019) Mobile application testing tools and their challenges: A comparative study. In iCoMET, pp 1\u20136","DOI":"10.1109\/ICOMET.2019.8673505"},{"key":"10726_CR3","unstructured":"Circleci (2021).https:\/\/circleci.com\/"},{"issue":"3","key":"10726_CR4","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1080\/17439760.2016.1262613","volume":"12","author":"V Clarke","year":"2017","unstructured":"Clarke V, Braun V (2017) Thematic analysis. J Posit Psychol 12(3):297\u2013298","journal-title":"J Posit Psychol"},{"key":"10726_CR5","unstructured":"Detox (2021). https:\/\/wix.github.io\/Detox\/"},{"key":"10726_CR6","unstructured":"Espresso (2021). https:\/\/developer.android.com\/training\/testing\/espresso"},{"key":"10726_CR7","doi-asserted-by":"crossref","unstructured":"Fazzini M, Xin Q, Orso A (2019) Automated api-usage update for android apps. In: ISSTA, pp 204\u2013215","DOI":"10.1145\/3293882.3330571"},{"key":"10726_CR8","unstructured":"F-Droid (2021) F-droid. https:\/\/f-droid.org"},{"key":"10726_CR9","unstructured":"Firebase test lab (2021). https:\/\/firebase.google.com\/docs\/test-lab"},{"key":"10726_CR10","unstructured":"Form: Survey on android app testing practices. https:\/\/forms.gle\/mFiMoJceHiLVnL2B8"},{"key":"10726_CR11","unstructured":"(2023) How many android users are there in the world? https:\/\/www.bankmycell.com\/blog\/how-many-android-users-are-there"},{"key":"10726_CR12","doi-asserted-by":"crossref","unstructured":"Haryono SA, Thung F, Lo D, Jiang L, Lawall J, Kang HJ, Serrano L, Muller G (2021) Androevolve: automated update for android deprecated-api usages. In: ICSE-Companion. IEEE, pp 1\u20134","DOI":"10.1109\/ICSE-Companion52605.2021.00021"},{"key":"10726_CR13","doi-asserted-by":"crossref","unstructured":"Hay R, Tripp O, Pistoia M (2015) Dynamic detection of inter-application communication vulnerabilities in android. In: ISSTA, pp 118\u2013128","DOI":"10.1145\/2771783.2771800"},{"key":"10726_CR14","unstructured":"Integration testing (2021). https:\/\/docs.flutter.dev\/testing\/integration-tests"},{"key":"10726_CR15","unstructured":"Jacoco java code coverage library (2018). https:\/\/www.jacoco.org\/jacoco\/"},{"key":"10726_CR16","unstructured":"Javaparser (2014). https:\/\/javaparser.org\/"},{"key":"10726_CR17","unstructured":"Jest (2021). https:\/\/jestjs.io\/docs\/tutorial-react-native"},{"key":"10726_CR18","unstructured":"Junit (2021). https:\/\/junit.org\/junit5\/"},{"issue":"9","key":"10726_CR19","first-page":"992","volume":"14","author":"M Kamran","year":"2016","unstructured":"Kamran M, Rashid J, Nisar MW (2016) Android fragmentation classification, causes, problems and solutions. Int J Comput Sci Inf Secur 14(9):992","journal-title":"Int J Comput Sci Inf Secur"},{"key":"10726_CR20","unstructured":"Kernel density estimation (2014). https:\/\/en.wikipedia.org\/wiki\/Kernel_density_estimation"},{"key":"10726_CR21","doi-asserted-by":"crossref","unstructured":"Kochhar PS, Xia X, Lo D (2019) Practitioners\u2019 views on good software testing practices. In: ICSE-SEIP. IEEE, pp 61\u201370","DOI":"10.1109\/ICSE-SEIP.2019.00015"},{"issue":"1","key":"10726_CR22","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1109\/TR.2018.2865733","volume":"68","author":"P Kong","year":"2019","unstructured":"Kong P, Li L, Gao J, Liu K, Bissyand\u00e9 TF, Klein J (2019) Automated testing of android apps: a systematic literature review. IEEE Trans Reliab 68(1):45\u201366","journal-title":"IEEE Trans Reliab"},{"key":"10726_CR23","unstructured":"Leakcanary (2021). https:\/\/square.github.io\/leakcanary\/"},{"key":"10726_CR24","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1016\/j.infsof.2017.04.001","volume":"88","author":"L Li","year":"2017","unstructured":"Li L, Bissyand\u00e9 TF, Papadakis M, Rasthofer S, Bartel A, Octeau D, Klein J, Traon L (2017) Static analysis of android apps: a systematic literature review. Inf Softw Technol 88:67\u201395","journal-title":"Inf Softw Technol"},{"key":"10726_CR25","doi-asserted-by":"crossref","unstructured":"Li L, Bissyand\u00e9 TF, Wang H, Klein J (2018) Cid: automating the detection of api-related compatibility issues in android apps. In: ISSTA, pp 153\u2013163","DOI":"10.1145\/3213846.3213857"},{"key":"10726_CR26","doi-asserted-by":"crossref","unstructured":"Lima DL, De\u00a0Souza Santos R, Garcia GP, Da\u00a0Silva SS, Fran\u00e7a C, Capretz LF (2023) Software testing and code refactoring: a survey with practitioners. In: ICSME. IEEE, pp 500\u2013507","DOI":"10.1109\/ICSME58846.2023.00064"},{"key":"10726_CR27","doi-asserted-by":"crossref","unstructured":"Linares-V\u00e1squez M, Moran K, Poshyvanyk D (2017) Continuous, evolutionary and large-scale: A new perspective for automated mobile app testing. In: 2017 IEEE international conference on software maintenance and evolution (ICSME), pp 399\u2013410. IEEE","DOI":"10.1109\/ICSME.2017.27"},{"key":"10726_CR28","doi-asserted-by":"crossref","unstructured":"Lo D, Nagappan N, Zimmermann T (2015) How practitioners perceive the relevance of software engineering research. In: FSE, pp 415\u2013425","DOI":"10.1145\/2786805.2786809"},{"key":"10726_CR29","doi-asserted-by":"publisher","first-page":"111664","DOI":"10.1016\/j.jss.2023.111664","volume":"200","author":"T Mahmud","year":"2023","unstructured":"Mahmud T, Che M, Yang G (2023) Analyzing the impact of API changes on android apps. J Syst Softw 200:111664","journal-title":"J Syst Softw"},{"issue":"7","key":"10726_CR30","doi-asserted-by":"publisher","first-page":"3857","DOI":"10.1109\/TSE.2023.3274153","volume":"49","author":"T Mahmud","year":"2023","unstructured":"Mahmud T, Che M, Yang G (2023) Detecting android API compatibility issues with API differences. IEEE Trans Softw Eng 49(7):3857\u20133871","journal-title":"IEEE Trans Softw Eng"},{"key":"10726_CR31","doi-asserted-by":"publisher","first-page":"107530","DOI":"10.1016\/j.infsof.2024.107530","volume":"175","author":"T Mahmud","year":"2024","unstructured":"Mahmud T, Che M, Yang G (2024) An empirical study on compatibility issues in android API field evolution. Inf Softw Technol 175:107530","journal-title":"Inf Softw Technol"},{"key":"10726_CR32","doi-asserted-by":"crossref","unstructured":"Mahmud T, Che M, Ngu AHH, Yang G (2024) An empirical investigation on android app testing practices. In: ISSRE. IEEE, pp 355\u2013366","DOI":"10.1109\/ISSRE62328.2024.00042"},{"key":"10726_CR33","doi-asserted-by":"crossref","unstructured":"Mahmud T, Che M, Yang G (2021) Android compatibility issue detection using api differences. In: SANER. IEEE, pp 480\u2013490","DOI":"10.1109\/SANER50967.2021.00051"},{"key":"10726_CR34","doi-asserted-by":"crossref","unstructured":"Mahmud T, Che M, Yang G (2022) Android api field evolution and its induced compatibility issues. In: ESEM, pp 34\u201344","DOI":"10.1145\/3544902.3546242"},{"key":"10726_CR35","doi-asserted-by":"crossref","unstructured":"Mahmud T, Che M, Yang G(2022) Acid: an api compatibility issue detector for android apps. In: ICSE-Companion. IEEE, pp 1\u20135","DOI":"10.1145\/3510454.3516854"},{"key":"10726_CR36","doi-asserted-by":"crossref","unstructured":"Mahmud T, Khan M, Rouijel J, Che M, Yang G (2021) API change impact analysis for android apps. In: COMPSAC. IEEE, pp 894\u2013903","DOI":"10.1109\/COMPSAC51774.2021.00122"},{"key":"10726_CR37","doi-asserted-by":"crossref","unstructured":"Mao K, Harman M, Jia Y (2016) Sapienz: multi-objective automated testing for android applications. In: ISSTA, pp 94\u2013105","DOI":"10.1145\/2931037.2931054"},{"key":"10726_CR38","doi-asserted-by":"crossref","unstructured":"Mirzaei N, Garcia J, Bagheri H, Sadeghi A, Malek S (2016) Reducing combinatorics in GUI testing of android applications. In: ICSE, pp 559\u2013570","DOI":"10.1145\/2884781.2884853"},{"key":"10726_CR39","unstructured":"Mockative (2023). https:\/\/github.com\/mockative\/mockative"},{"key":"10726_CR40","unstructured":"Mockito (2021). https:\/\/site.mockito.org\/"},{"key":"10726_CR41","unstructured":"Mockk (2021). https:\/\/mockk.io\/"},{"key":"10726_CR42","doi-asserted-by":"publisher","first-page":"100963","DOI":"10.1016\/j.cola.2020.100963","volume":"58","author":"M Ozkaya","year":"2020","unstructured":"Ozkaya M, Erata F (2020) Understanding practitioners\u2019 challenges on software modeling: a survey. J Comput Lang 58:100963","journal-title":"J Comput Lang"},{"key":"10726_CR43","unstructured":"Appium (2021). https:\/\/appium.io\/docs\/en\/2.2\/"},{"key":"10726_CR44","doi-asserted-by":"crossref","unstructured":"Pecorelli F, Catolino G, Ferrucci F, De\u00a0Lucia A, Palomba F (2022) Software testing and android applications: a large-scale empirical study. Empir Softw Eng","DOI":"10.1007\/s10664-021-10059-5"},{"key":"10726_CR45","doi-asserted-by":"crossref","unstructured":"Peruma A, Almalki K, Newman CD, Mkaouer MW, Ouni A, Palomba F (2020) Tsdetect: an open source test smells detection tool. In: ESEC\/FSE, pp 1650\u20131654","DOI":"10.1145\/3368089.3417921"},{"key":"10726_CR46","unstructured":"Postman (2019). https:\/\/www.postman.com\/"},{"key":"10726_CR47","unstructured":"Profile your app performance (2021). https:\/\/developer.android.com\/studio\/profile"},{"key":"10726_CR48","doi-asserted-by":"crossref","unstructured":"Raulamo-Jurvanen P, Hosio S, M\u00e4ntyl\u00e4 MV (2019) Practitioner evaluations on software testing tools. In: EASE, pp 57\u201366","DOI":"10.1145\/3319008.3319018"},{"key":"10726_CR49","unstructured":"Replication package. https:\/\/github.com\/TSUMahmud\/An-Empirical-Investigation-on-Android-App-Testing-Practices\/"},{"key":"10726_CR50","doi-asserted-by":"publisher","first-page":"106488","DOI":"10.1016\/j.infsof.2020.106488","volume":"131","author":"K Rindell","year":"2021","unstructured":"Rindell K, Ruohonen J, Holvitie J, Hyrynsalmi S, Lepp\u00e4nen V (2021) Security in agile software development: a practitioner survey. Inf Softw Technol 131:106488","journal-title":"Inf Softw Technol"},{"key":"10726_CR51","unstructured":"Robolectric (2021). https:\/\/robolectric.org\/"},{"key":"10726_CR52","doi-asserted-by":"crossref","unstructured":"Sasnauskas R, Regehr J (2014) Intent fuzzer: crafting intents of death. In: WODA+PERTEA, pp 1\u20135","DOI":"10.1145\/2632168.2632169"},{"key":"10726_CR53","unstructured":"Sauce labs (2021). https:\/\/saucelabs.com\/products\/mobile-testing"},{"key":"10726_CR54","unstructured":"Selendroid (2020). https:\/\/github.com\/selendroid\/selendroid"},{"key":"10726_CR55","doi-asserted-by":"crossref","unstructured":"Song W, Qian X, Huang J (2017) Ehbdroid: beyond GUI testing for android applications. In: ASE. IEEE, pp 27\u201337","DOI":"10.1109\/ASE.2017.8115615"},{"key":"10726_CR56","doi-asserted-by":"crossref","unstructured":"Sun X, Chen X, Zhao Y, Liu P, Grundy J, Li L (2023) Mining android API usage to generate unit test cases for pinpointing compatibility issues. In: ASE","DOI":"10.1145\/3551349.3561151"},{"key":"10726_CR57","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1007\/s11219-018-9418-6","volume":"27","author":"P Tramontana","year":"2019","unstructured":"Tramontana P, Amalfitano D, Amatucci N, Fasolino AR (2019) Automated functional testing of mobile applications: a systematic mapping study. Softw Qual J 27:149\u2013201","journal-title":"Softw Qual J"},{"key":"10726_CR58","doi-asserted-by":"crossref","unstructured":"Vilkomir S, Amstutz B (2014) Using combinatorial approaches for testing mobile applications. In: ICST workshops","DOI":"10.1109\/ICSTW.2014.9"},{"key":"10726_CR59","doi-asserted-by":"crossref","unstructured":"Wei L, Liu Y, Cheung S (2016) Taming android fragmentation: characterizing and detecting compatibility issues for android apps. In: ASE, pp 226\u2013237","DOI":"10.1145\/2970276.2970312"},{"key":"10726_CR60","doi-asserted-by":"publisher","DOI":"10.1109\/MOBS.2013.6614215","volume-title":"Testing for poor responsiveness in android applications","author":"S Yang","year":"2013","unstructured":"Yang S, Yan D, Rountev A (2013) Testing for poor responsiveness in android applications. In MOBS, IEEE"},{"key":"10726_CR61","doi-asserted-by":"crossref","unstructured":"Yan D, Yang S, Rountev A (2013) Systematic testing for resource leaks in android applications. In: ISSRE. IEEE, pp 411\u2013420","DOI":"10.1109\/ISSRE.2013.6698894"},{"key":"10726_CR62","doi-asserted-by":"crossref","unstructured":"Zhang T, Gao J, Cheng J, Uehara T (2015) Compatibility testing service for mobile applications. In: SOSE, pp 179\u2013186","DOI":"10.1109\/SOSE.2015.35"},{"key":"10726_CR63","doi-asserted-by":"crossref","unstructured":"Zhang T, Gao J, Cheng J, Uehara T (2015) Compatibility testing service for mobile applications. In: SOSE, pp 179\u2013186","DOI":"10.1109\/SOSE.2015.35"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-025-10726-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-025-10726-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-025-10726-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T02:41:55Z","timestamp":1762310515000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-025-10726-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,22]]},"references-count":63,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,12]]}},"alternative-id":["10726"],"URL":"https:\/\/doi.org\/10.1007\/s10664-025-10726-x","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,22]]},"assertion":[{"value":"5 March 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 August 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 September 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"As this survey is truly anonymous and non-sensitive and for general research purposes, it meets the IRB exemption criteria.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"The authors declare that they have no conflict of interest.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}}],"article-number":"163"}}