{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T19:20:36Z","timestamp":1792783236106,"version":"4.5.0"},"reference-count":74,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T00:00:00Z","timestamp":1788134400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T00:00:00Z","timestamp":1788134400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Politecnico di Torino","id":[{"id":"https:\/\/ror.org\/00bgk9508","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Software Qual J"],"published-print":{"date-parts":[[2026,9]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Testing of mobile applications is a crucial activity that aims to reveal bugs and defects affecting software running on our smartphones. Usually, mobile applications feature complex Graphical User Interfaces (GUIs) as the only interface with the human user, therefore proper End-to-End testing of mobile applications should also include GUI testing. In recent years, several automated testing methods have been proposed for mobile GUI testing; however, due to the dynamic and interactive nature of GUIs, they often fail to create realistic use cases. Exploratory testing is a popular GUI testing technique that involves testers freely exploring the application under test. This technique heavily relies upon testers\u2019 experience and creativity, and is negatively influenced by human factors like stress and boredom, which are likely to appear in unmotivated testers. Gamification, i.e., the introduction of game design mechanics to tasks of other nature, has recently gained much popularity in business contexts, being applied to a wide range of software-related activities. In this article, we propose a gamification approach to mobile Exploratory Testing to improve the effectiveness, efficiency, and engagement of the software testers. We measure the impact of gamification with an in-vivo experiment involving students from a master\u2019s degree software engineering course, adopting a fully-randomized experiment design. On the efficiency side, results show a positive impact of gamification on the explored pages \u2013 a measure of effectiveness \u2013 and on the explored pages per unit of time \u2013 which we adopted as an efficiency measure. Moreover, a final survey presented to students reveals that most of them believe that it is a good idea to use gamification in Exploratory Testing, and that they would use gamification in future if they had to do Exploratory Testing. Finally, the majority of participants report having fun while using the system. Our results highlight that gamification features can have a significant impact if introduced in a classical testing environment, possibly increasing effectiveness, efficiency and perceived engagement of the testers. We believe this research could serve as a starting point for researchers approaching the mobile testing environment, as well as industry practitioners working in the field.<\/jats:p>","DOI":"10.1007\/s11219-026-09781-4","type":"journal-article","created":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T14:45:11Z","timestamp":1788187511000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["GAppium experiment: Applying gamification to mobile GUI testing to increase effectiveness, efficiency, engagement"],"prefix":"10.1007","volume":"34","author":[{"given":"Lorenzo","family":"Laudadio","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Riccardo","family":"Coppola","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marco","family":"Torchiano","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,8,31]]},"reference":[{"key":"9781_CR1","doi-asserted-by":"publisher","unstructured":"Alegroth, E., Gao, Z., Oliveira, R., & Memon, A. (2015). Conceptualization and evaluation of component-based testing unified with visual gui testing: An empirical study. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST) (pp. 1\u201310). https:\/\/doi.org\/10.1109\/ICST.2015.7102584","DOI":"10.1109\/ICST.2015.7102584"},{"issue":"5","key":"9781_CR2","doi-asserted-by":"publisher","first-page":"502","DOI":"10.1111\/opo.12131","volume":"34","author":"RA Armstrong","year":"2014","unstructured":"Armstrong, R. A. (2014). When to use the Bonferroni correction. Ophthalmic and Physiological Optics, 34(5), 502\u2013508. https:\/\/doi.org\/10.1111\/opo.12131","journal-title":"Ophthalmic and Physiological Optics"},{"key":"9781_CR3","unstructured":"Bach, J. (2019). Exploratory Testing. https:\/\/www.satisfice.com\/exploratory-testing Accessed 16 Oct 2025"},{"issue":"6","key":"9781_CR4","doi-asserted-by":"publisher","first-page":"728","DOI":"10.1109\/TSE.1984.5010301","volume":"SE\u201310","author":"VR Basili","year":"1984","unstructured":"Basili, V. R., & Weiss, D. M. (1984). A Methodology for Collecting Valid Software Engineering Data. IEEE Transactions on Software Engineering, SE\u201310(6), 728\u2013738. https:\/\/doi.org\/10.1109\/TSE.1984.5010301","journal-title":"IEEE Transactions on Software Engineering"},{"key":"9781_CR5","doi-asserted-by":"publisher","unstructured":"Baxter, K., Courage, C., & Caine, K. (2015). Chapter 10 - surveys. In K. Baxter, C. Courage, & K. Caine (eds.), Understanding Your Users (Second Edition) (pp. 264\u2013301), Second edition edn. Interactive Technologies. Morgan Kaufmann, Boston. https:\/\/doi.org\/10.1016\/B978-0-12-800232-2.00010-9.","DOI":"10.1016\/B978-0-12-800232-2.00010-9"},{"key":"9781_CR6","doi-asserted-by":"publisher","first-page":"111647","DOI":"10.1016\/j.jss.2023.111647","volume":"200","author":"R Blanco","year":"2023","unstructured":"Blanco, R., Trinidad, M., Su\u00e1rez-Cabal, M. J., Calder\u00f3n, A., Ruiz, M., & Tuya, J. (2023). Can gamification help in software testing education? Findings from an empirical study. Journal of Systems and Software, 200, 111647. https:\/\/doi.org\/10.1016\/j.jss.2023.111647","journal-title":"Journal of Systems and Software"},{"key":"9781_CR7","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1136\/bmj.310.6973.170","volume":"310","author":"JM Bland","year":"1995","unstructured":"Bland, J. M., & Altman, D. G. (1995). Multiple significance tests: The bonferroni method. BMJ, 310, 170. https:\/\/doi.org\/10.1136\/bmj.310.6973.170","journal-title":"BMJ"},{"issue":"1","key":"9781_CR8","doi-asserted-by":"publisher","first-page":"923","DOI":"10.1177\/1071181321651311","volume":"65","author":"S Brantley","year":"2021","unstructured":"Brantley, S., Wilkinson, M., & Feng, J. (2021). Beat the bots: Exploring the effects of placebo manipulation on performance during video gameplay. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 65(1), 923\u2013927. https:\/\/doi.org\/10.1177\/1071181321651311","journal-title":"Proceedings of the Human Factors and Ergonomics Society Annual Meeting"},{"key":"9781_CR9","unstructured":"Brooke, J. (1996). SUS - a quick and dirty usability scale (pp. 189\u2013194)"},{"issue":"2","key":"9781_CR10","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1109\/TR.2018.2799957","volume":"67","author":"M Bures","year":"2018","unstructured":"Bures, M., Frajtak, K., & Ahmed, B. S. (2018). Tapir: Automation support of exploratory testing using model reconstruction of the system under test. IEEE Transactions on Reliability, 67(2), 557\u2013580. https:\/\/doi.org\/10.1109\/TR.2018.2799957","journal-title":"IEEE Transactions on Reliability"},{"key":"9781_CR11","unstructured":"Chou, Y. K. (2015). Actionable gamification: Beyond points, badges, and leaderboards. Createspace Independent Publishing Platform, South Carolina, USA. https:\/\/books.google.it\/books?id=jFWQrgEACAAJ"},{"key":"9781_CR12","doi-asserted-by":"publisher","unstructured":"Coppola, R., Ardito, L., Torchiano, M., & Al\u00e8groth, E. (2020). Translation from visual to layout-based android test cases: a proof of concept. In 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 74\u201383). https:\/\/doi.org\/10.1109\/ICSTW50294.2020.00027","DOI":"10.1109\/ICSTW50294.2020.00027"},{"issue":"2","key":"9781_CR13","doi-asserted-by":"publisher","first-page":"322","DOI":"10.1109\/TSE.2023.3348036","volume":"50","author":"R Coppola","year":"2024","unstructured":"Coppola, R., Fulcini, T., Ardito, L., Torchiano, M., & Al\u00e8groth, E. (2024). On effectiveness and efficiency of gamified exploratory GUI testing. IEEE Transactions on Software Engineering, 50(2), 322\u2013337. https:\/\/doi.org\/10.1109\/TSE.2023.3348036","journal-title":"IEEE Transactions on Software Engineering"},{"key":"9781_CR14","doi-asserted-by":"publisher","unstructured":"Costa, I. E. F., & Oliveira, S. R. B. (2021). A study on assets applied in exploratory test design and execution: An interview application. In 2021 IEEE Frontiers in Education Conference (FIE) (pp. 1\u20138). https:\/\/doi.org\/10.1109\/FIE49875.2021.9637359","DOI":"10.1109\/FIE49875.2021.9637359"},{"key":"9781_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.infsof.2016.01.002","volume":"73","author":"A Deak","year":"2016","unstructured":"Deak, A., St\u00e5lhane, T., & Sindre, G. (2016). Challenges and strategies for motivating software testing personnel. Information and software Technology, 73, 1\u201315.","journal-title":"Information and software Technology"},{"key":"9781_CR16","doi-asserted-by":"publisher","unstructured":"Deci, E. L., & Ryan, R. M. (1985). Intrinsic Motivation and Self-Determination in Human Behavior. Springer, Boston, MA. https:\/\/doi.org\/10.1007\/978-1-4899-2271-7","DOI":"10.1007\/978-1-4899-2271-7"},{"key":"9781_CR17","unstructured":"Deterding, S., Khaled, R., Nacke, L. E., & Dixon, D. (2011). Gamification: Toward a definition. In CHI 2011 Gamification Workshop Proceedings (Vol. 12, pp. 1\u201379). Vancouver."},{"issue":"3","key":"9781_CR18","doi-asserted-by":"publisher","first-page":"1754","DOI":"10.1002\/stvr.1754","volume":"31","author":"S Di Martino","year":"2021","unstructured":"Di Martino, S., Fasolino, A. R., Starace, L. L. L., & Tramontana, P. (2021). Comparing the effectiveness of capture and replay against automatic input generation for Android graphical user interface testing. Software Testing, Verification and Reliability, 31(3), 1754. https:\/\/doi.org\/10.1002\/stvr.1754","journal-title":"Software Testing, Verification and Reliability"},{"key":"9781_CR19","doi-asserted-by":"publisher","unstructured":"Evers, B., Derakhshanfar, P., Devroey, X., & Zaidman, A. (2020). Commonality-driven unit test generation. In Search-Based Software Engineering: 12th International Symposium (pp. 121\u2013136), SSBSE 2020, Bari, Italy, October 7\u20138, 2020, Proceedings. Springer, Berlin, Heidelberg. https:\/\/doi.org\/10.1007\/978-3-030-59762-7_9 .","DOI":"10.1007\/978-3-030-59762-7_9"},{"key":"9781_CR20","doi-asserted-by":"publisher","unstructured":"Fasolino, A. R., Accetto, C. M., & Tramontana, P. (2024). Testing robot challenge: A serious game for testing learning. In Proceedings of the 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation (pp. 26\u201329). Gamify 2024. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3678869.3685686 . https:\/\/dl.acm.org\/doi\/10.1145\/3678869.3685686 Accessed 2025-02-05","DOI":"10.1145\/3678869.3685686"},{"key":"9781_CR21","unstructured":"F-Droid - Free and Open Source Android App Repository. https:\/\/f-droid.org\/ Accessed 28 OCt 2025"},{"key":"9781_CR22","unstructured":"federicoiosue: Releases \u00b7 federicoiosue\/Omni-Notes. https:\/\/github.com\/federicoiosue\/Omni-Notes\/releases Accessed 05 Nov 2025"},{"issue":"2","key":"9781_CR23","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1177\/001872675400700202","volume":"7","author":"L Festinger","year":"1954","unstructured":"Festinger, L. (1954). A theory of social comparison processes. Human Relations, 7(2), 117\u2013140. https:\/\/doi.org\/10.1177\/001872675400700202","journal-title":"Human Relations"},{"key":"9781_CR24","doi-asserted-by":"publisher","unstructured":"Fraser, G., Gambi, A., Kreis, M., & Rojas, J. M. (2019). Gamifying a Software Testing Course with Code Defenders. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (pp. 571\u2013577). SIGCSE \u201919. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3287324.3287471","DOI":"10.1145\/3287324.3287471"},{"issue":"1","key":"9781_CR25","doi-asserted-by":"publisher","first-page":"59","DOI":"10.3102\/00346543074001059","volume":"74","author":"JA Fredricks","year":"2004","unstructured":"Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59\u2013109. https:\/\/doi.org\/10.3102\/00346543074001059","journal-title":"Review of Educational Research"},{"key":"9781_CR26","doi-asserted-by":"publisher","unstructured":"Fulcini, T., & Ardito, L. (2022). Gamified exploratory GUI testing of web applications: A preliminary evaluation. In 2022 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 215\u2013222). https:\/\/doi.org\/10.1109\/ICSTW55395.2022.00045. https:\/\/ieeexplore.ieee.org\/document\/9787968 Accessed 13 Jun 2024","DOI":"10.1109\/ICSTW55395.2022.00045"},{"issue":"14s","key":"9781_CR27","doi-asserted-by":"publisher","first-page":"310","DOI":"10.1145\/3582273","volume":"55","author":"T Fulcini","year":"2023","unstructured":"Fulcini, T., Coppola, R., Ardito, L., & Torchiano, M. (2023). A review on tools, mechanics, benefits, and challenges of gamified software testing. ACM Computing Surveys, 55(14s), 310\u2013131037. https:\/\/doi.org\/10.1145\/3582273","journal-title":"ACM Computing Surveys"},{"key":"9781_CR28","doi-asserted-by":"publisher","unstructured":"Garaccione, G., Fulcini, T., Stefanut\u00a0Bodnarescul, P., Coppola, R., & Ardito, L. (2024). Gamified GUI testing with Selenium in the IntelliJ IDE: A Prototype Plugin. In Proceedings of the 1st ACM\/IEEE Workshop on Integrated Development Environments (pp. 76\u201380). IDE \u201924. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3643796.3648459 Accessed 04 May 2026","DOI":"10.1145\/3643796.3648459"},{"key":"9781_CR29","doi-asserted-by":"publisher","unstructured":"Gebizli, C.c., & S\u00f6zer, H. (2017). Automated refinement of models for model-based testing using exploratory testing. Software Quality Journal, 25(3), 979\u20131005. https:\/\/doi.org\/10.1007\/s11219-016-9338-2","DOI":"10.1007\/s11219-016-9338-2"},{"key":"9781_CR30","doi-asserted-by":"publisher","first-page":"26416","DOI":"10.1109\/ACCESS.2018.2834957","volume":"6","author":"AN Ghazi","year":"2018","unstructured":"Ghazi, A. N., Petersen, K., Bjarnason, E., & Runeson, P. (2018). Levels of exploration in exploratory testing: From freestyle to fully scripted. IEEE Access, 6, 26416\u201326423. https:\/\/doi.org\/10.1109\/ACCESS.2018.2834957","journal-title":"IEEE Access"},{"key":"9781_CR31","doi-asserted-by":"publisher","unstructured":"Heasman, B., & Gillespie, A. (2019). Participants over-estimate how helpful they are in a two-player game scenario toward an artificial confederate that discloses a diagnosis of autism. Frontiers in Psychology, 10 - 2019. https:\/\/doi.org\/10.3389\/fpsyg.2019.01349","DOI":"10.3389\/fpsyg.2019.01349"},{"issue":"2","key":"9781_CR32","first-page":"405","volume":"29","author":"PC Jordan","year":"1986","unstructured":"Jordan, P. C. (1986). Effects of an extrinsic reward on intrinsic motivation: A field experiment. The Academy of Management Journal, 29(2), 405\u2013412.","journal-title":"The Academy of Management Journal"},{"key":"9781_CR33","doi-asserted-by":"crossref","unstructured":"Laudadio, L., Coppola, R., Torchiano, M., & Fulcini, T. (2025). GAppium: A Framework to Enact Gamification Mechanics in Appium Inspector.","DOI":"10.1109\/Gamify66603.2025.00006"},{"key":"9781_CR34","doi-asserted-by":"publisher","unstructured":"Leveau, J., Blanc, X., R\u00e9veill\u00e8re, L., Falleri, J.-R., & Rouvoy, R. (2020). Fostering the diversity of exploratory testing in web applications. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (pp. 164\u2013174). ICST. https:\/\/doi.org\/10.1109\/ICST46399.2020.00026","DOI":"10.1109\/ICST46399.2020.00026"},{"issue":"5","key":"9781_CR35","doi-asserted-by":"publisher","first-page":"1827","DOI":"10.1002\/stvr.1827","volume":"32","author":"J Leveau","year":"2022","unstructured":"Leveau, J., Blanc, X., R\u00e9veill\u00e8re, L., Falleri, J.-R., & Rouvoy, R. (2022). Fostering the diversity of exploratory testing in web applications. Software Testing, Verification and Reliability, 32(5), 1827. https:\/\/doi.org\/10.1002\/stvr.1827","journal-title":"Software Testing, Verification and Reliability"},{"key":"9781_CR36","unstructured":"Ligi (2025) ligi\/PassAndroid. https:\/\/github.com\/ligi\/PassAndroid Accessed 05 Nov 2025"},{"key":"9781_CR37","doi-asserted-by":"publisher","unstructured":"Long, Z., Wu, G., Chen, X., Chen, W., & Wei, J. (2020). WebRR: self-replay enhanced robust record\/replay for web application testing. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (pp. 1498\u20131508). ESEC\/FSE 2020. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3368089.3417069","DOI":"10.1145\/3368089.3417069"},{"issue":"6","key":"9781_CR38","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1007\/s10664-025-10726-x","volume":"30","author":"T Mahmud","year":"2025","unstructured":"Mahmud, T., Che, M., Ngu, A., & Yang, G. (2025). Why android app testing falls short: Empirical insights from open-source projects and a practitioner survey. Empirical Software Engineering, 30(6), 163. https:\/\/doi.org\/10.1007\/s10664-025-10726-x","journal-title":"Empirical Software Engineering"},{"key":"9781_CR39","doi-asserted-by":"publisher","first-page":"110890","DOI":"10.1016\/j.jss.2020.110890","volume":"174","author":"T M\u00e5rtensson","year":"2021","unstructured":"M\u00e5rtensson, T., St\u00e5hl, D., Martini, A., & Bosch, J. (2021). Efficient and effective exploratory testing of large-scale software systems. Journal of Systems and Software, 174, 110890. https:\/\/doi.org\/10.1016\/j.jss.2020.110890","journal-title":"Journal of Systems and Software"},{"issue":"4","key":"9781_CR40","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1109\/TSE.1976.233837","volume":"SE\u20132","author":"TJ McCabe","year":"1976","unstructured":"McCabe, T. J. (1976). A complexity measure. IEEE Transactions on Software Engineering, SE\u20132(4), 308\u2013320. https:\/\/doi.org\/10.1109\/TSE.1976.233837","journal-title":"IEEE Transactions on Software Engineering"},{"key":"9781_CR41","doi-asserted-by":"publisher","unstructured":"Mikhaltsov, D., & Sorokin, K. (2022). Detecting anomalous device loads during exploratory testing of mobile applications. In 2022 Ivannikov Ispras Open Conference (ISPRAS) (pp. 43\u201349). https:\/\/doi.org\/10.1109\/ISPRAS57371.2022.10076851","DOI":"10.1109\/ISPRAS57371.2022.10076851"},{"key":"9781_CR42","doi-asserted-by":"publisher","unstructured":"Mukherjee, R., Pati, S.K., & Banerjee, A. (2022). Chapter 2 - Performance tuning of Android applications using clustering and optimization heuristics. In S. De, S. Dey, S. Bhattacharyya, & S. Bhatia (eds.), Advanced Data Mining Tools and Methods for Social Computing. Hybrid Computational Intelligence for Pattern Analysis (pp. 27\u201350). Academic Press, Cambridge, Massachusetts, USA. https:\/\/doi.org\/10.1016\/B978-0-32-385708-6.00009-6","DOI":"10.1016\/B978-0-32-385708-6.00009-6"},{"key":"9781_CR43","doi-asserted-by":"publisher","unstructured":"Nass, M., Al\u00e9groth, E., & Feldt, R. (2019). Augmented Testing: Industry Feedback To Shape a New Testing Technology. In 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 176\u2013183). https:\/\/doi.org\/10.1109\/ICSTW.2019.00048","DOI":"10.1109\/ICSTW.2019.00048"},{"key":"9781_CR44","doi-asserted-by":"publisher","unstructured":"Negara, S., Esfahani, N., & Buse, R. (2019). Practical android test recording with espresso test recorder. In 2019 IEEE\/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) (pp. 193\u2013202). https:\/\/doi.org\/10.1109\/ICSE-SEIP.2019.00029","DOI":"10.1109\/ICSE-SEIP.2019.00029"},{"key":"9781_CR45","doi-asserted-by":"publisher","unstructured":"Neri, G., Marchand, R., & Walkinshaw, N. (2025). Exploratory software testing in scrum: A qualitative study. In S. Peter, M. Kropp, A. Aguiar, C. Anslow, M. I. Lunesu, & A. Pinna (eds.), Agile Processes in Software Engineering and Extreme Programming (pp. 160\u2013175). Springer, Cham. https:\/\/doi.org\/10.1007\/978-3-031-94544-1_11","DOI":"10.1007\/978-3-031-94544-1_11"},{"key":"9781_CR46","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1016\/j.infsof.2014.08.007","volume":"57","author":"O Pedreira","year":"2015","unstructured":"Pedreira, O., Garc\u00eda, F., Brisaboa, N., & Piattini, M. (2015). Gamification in software engineering \u2013 A systematic mapping. Information and Software Technology, 57, 157\u2013168. https:\/\/doi.org\/10.1016\/j.infsof.2014.08.007","journal-title":"Information and Software Technology"},{"issue":"7139","key":"9781_CR47","doi-asserted-by":"publisher","first-page":"1236","DOI":"10.1136\/bmj.316.7139.1236","volume":"316","author":"TV Perneger","year":"1998","unstructured":"Perneger, T. V. (1998). What\u2019s wrong with Bonferroni adjustments. BMJ, 316(7139), 1236\u20131238. https:\/\/doi.org\/10.1136\/bmj.316.7139.1236","journal-title":"BMJ"},{"key":"9781_CR48","doi-asserted-by":"publisher","unstructured":"Pfahl, D., Yin, H., M\u00e4ntyl\u00e4, M. V., & M\u00fcnch, J. (2014). How is exploratory testing used? A state-of-the-practice survey. In Proceedings of the 8th ACM\/IEEE International Symposium on Empirical Software Engineering And Measurement. ESEM \u201914 (pp. 1\u201310). New York, NY, USA: Association for Computing Machinery. https:\/\/doi.org\/10.1145\/2652524.2652531","DOI":"10.1145\/2652524.2652531"},{"key":"9781_CR49","unstructured":"Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., Heisterkamp, S., authors, E., Ranke, J., Van\u00a0Willigen, B., & Team, R. C. (2025). Nlme: Linear and Nonlinear Mixed Effects Models. https:\/\/cran.r-project.org\/web\/packages\/nlme\/index.html Accessed 29 Oct 2025"},{"key":"9781_CR50","unstructured":"Plot Likert-type items with \u2018gglikert()\u2018. https:\/\/larmarange.github.io\/ggstats\/articles\/gglikert.html Accessed 29 Oct 2025"},{"key":"9781_CR51","doi-asserted-by":"publisher","first-page":"110870","DOI":"10.1016\/j.jss.2020.110870","volume":"173","author":"DDP Porto","year":"2021","unstructured":"Porto, D. D. P., Jesus, G. M. D., Ferrari, F. C., & Fabbri, S. C. P. F. (2021). Initiatives and challenges of using gamification in software engineering: A systematic mapping. Journal of Systems and Software, 173, 110870. https:\/\/doi.org\/10.1016\/j.jss.2020.110870","journal-title":"Journal of Systems and Software"},{"key":"9781_CR52","doi-asserted-by":"publisher","first-page":"106392","DOI":"10.1016\/j.chb.2020.106392","volume":"110","author":"L-M Putz","year":"2020","unstructured":"Putz, L.-M., Hofbauer, F., & Treiblmaier, H. (2020). Can gamification help to improve education? Findings from a longitudinal study. Computers in Human Behavior, 110, 106392. https:\/\/doi.org\/10.1016\/j.chb.2020.106392","journal-title":"Computers in Human Behavior"},{"key":"9781_CR53","unstructured":"Quality management systems \u2014 Requirements. Standard, International Organization for Standardization, Geneva, CH (2015)"},{"key":"9781_CR54","unstructured":"R Core Team (2025) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. R Foundation for Statistical Computing. https:\/\/www.R-project.org\/"},{"key":"9781_CR55","unstructured":"RaiMan\u2019s SikuliX (2025). http:\/\/sikulix.com\/index.html Accessed 17 Oct 2025"},{"key":"9781_CR56","doi-asserted-by":"publisher","unstructured":"Ren, W. (2023). Gamification in Test-Driven Development Practice. In Proceedings of the 2nd International Workshop on Gamification in Software Development, Verification, And Validation (pp. 38\u201346). Gamify 2023. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3617553.3617889","DOI":"10.1145\/3617553.3617889"},{"key":"9781_CR57","doi-asserted-by":"publisher","unstructured":"Romdhana, A., Merlo, A., Ceccato, M., & Tonella, P. (2022). Deep reinforcement learning for black-box testing of android apps. ACM Transactions on Software Engineering and Methodology, 31(4). https:\/\/doi.org\/10.1145\/3502868","DOI":"10.1145\/3502868"},{"key":"9781_CR58","doi-asserted-by":"publisher","unstructured":"Shamshiri, S., Rojas, J. M., Galeotti, J. P., Walkinshaw, N., & Fraser, G. (2018). How do automatically generated unit tests influence software maintenance? In IEEE 11th International Conference on Software Testing, Verification and Validation (pp. 250\u2013261). ICST. https:\/\/doi.org\/10.1109\/ICST.2018.00033","DOI":"10.1109\/ICST.2018.00033"},{"issue":"2","key":"9781_CR59","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3530796","volume":"5","author":"M Stoeve","year":"2022","unstructured":"Stoeve, M., Wirth, M., Farlock, R., Antunovic, A., M\u00fcller, V., & Eskofier, B. M. (2022). Eye tracking-based stress classification of athletes in virtual reality. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 5(2), 1\u201317. https:\/\/doi.org\/10.1145\/3530796","journal-title":"Proceedings of the ACM on Computer Graphics and Interactive Techniques"},{"key":"9781_CR60","doi-asserted-by":"publisher","unstructured":"Straubinger, P., & Fraser, G. (2023). A survey on what developers think about testing. In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) (pp. 80\u201390 ). https:\/\/doi.org\/10.1109\/ISSRE59848.2023.00075. https:\/\/ieeexplore.ieee.org\/abstract\/document\/10301252 Accessed 14 Nov 2024","DOI":"10.1109\/ISSRE59848.2023.00075"},{"key":"9781_CR61","doi-asserted-by":"publisher","unstructured":"Straubinger, P., & Fraser, G. (2024). Engaging developers in exploratory unit testing through gamification. In Proceedings of the 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation (pp. 2\u20139). Gamify 2024. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3678869.3685683","DOI":"10.1145\/3678869.3685683"},{"key":"9781_CR62","doi-asserted-by":"publisher","unstructured":"Straubinger, P., Fulcini, T., Garaccione, G., Ardito, L., & Fraser, G. (2025). Gamifying testing in intellij: A replicability study. Proceedings of the ACM on Software Engineering, 2(ISSTA). https:\/\/doi.org\/10.1145\/3728983","DOI":"10.1145\/3728983"},{"issue":"2","key":"9781_CR63","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1207\/s15516709cog1202_4","volume":"12","author":"J Sweller","year":"1988","unstructured":"Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257\u2013285. https:\/\/doi.org\/10.1207\/s15516709cog1202_4","journal-title":"Cognitive Science"},{"key":"9781_CR64","volume-title":"Introduction to Data Mining (2nd Edition)","author":"P-N Tan","year":"2018","unstructured":"Tan, P.-N., Steinbach, M., Karpatne, A., & Kumar, V. (2018). Introduction to Data Mining (2nd Edition) (2nd ed.). London, England, UK: Pearson.","edition":"2"},{"key":"9781_CR65","unstructured":"terryyin: terryyin\/lizard. https:\/\/github.com\/terryyin\/lizard Accessed 11 May 2026"},{"key":"9781_CR66","doi-asserted-by":"publisher","unstructured":"Valle, P. H. D., Vilela, R. F., & Hernandes, E. C. M. (2021). Does Gamification Improve the Training of Software Testers? A Preliminary Study from the Industry Perspective. In Proceedings of the XIX Brazilian Symposium on Software Quality (pp. 1\u201310). SBQS \u201920. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3439961.3440004","DOI":"10.1145\/3439961.3440004"},{"issue":"2","key":"9781_CR67","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1109\/TSE.2015.2467378","volume":"42","author":"S Vegas","year":"2016","unstructured":"Vegas, S., Apa, C., & Juristo, N. (2016). Crossover designs in software engineering experiments: Benefits and perils. IEEE Transactions on Software Engineering, 42(2), 120\u2013135. https:\/\/doi.org\/10.1109\/TSE.2015.2467378","journal-title":"IEEE Transactions on Software Engineering"},{"key":"9781_CR68","doi-asserted-by":"publisher","unstructured":"Vorderer, P., Hartmann, T., & Klimmt, C. (2003). Explaining the enjoyment of playing video games: The role of competition. https:\/\/doi.org\/10.1145\/958720.958735","DOI":"10.1145\/958720.958735"},{"key":"9781_CR69","doi-asserted-by":"publisher","unstructured":"Wang, Q., Brun, Y., & Orso, A. (2017). Behavioral execution comparison: Are tests representative of field behavior? In 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST) (pp. 321\u2013332). https:\/\/doi.org\/10.1109\/ICST.2017.36","DOI":"10.1109\/ICST.2017.36"},{"key":"9781_CR70","unstructured":"Washizaki, H. (2024). Guide to the Software Engineering Body of Knowledge (SWEBOK): Version 4.0, 4th edn. IEEE Computer Society Press."},{"key":"9781_CR71","unstructured":"Welcome - Appium Inspector. https:\/\/appium.github.io\/appium-inspector\/latest Accessed 28 Oct 2025"},{"key":"9781_CR72","unstructured":"Wikipedia (2025) Likert Scale. https:\/\/en.wikipedia.org\/wiki\/Likert_scale"},{"key":"9781_CR73","doi-asserted-by":"publisher","unstructured":"Winkler, A., & Hermann, C. (2019). Placebo- and nocebo-effects in cognitive neuroenhancement: When expectation shapes perception. Frontiers in Psychiatry, 10. https:\/\/doi.org\/10.3389\/fpsyt.2019.00498","DOI":"10.3389\/fpsyt.2019.00498"},{"key":"9781_CR74","doi-asserted-by":"publisher","unstructured":"Wohlin, C., Runeson, P., H\u00f6st, M., Ohlsson, M. C., Regnell, B., & Wessl\u00e9n, A. (2012). Experimentation in Software Engineering. Springer, Berlin, Heidelberg. https:\/\/doi.org\/10.1007\/978-3-642-29044-2","DOI":"10.1007\/978-3-642-29044-2"}],"container-title":["Software Quality Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11219-026-09781-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11219-026-09781-4","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11219-026-09781-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T10:59:50Z","timestamp":1790852390000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11219-026-09781-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,31]]},"references-count":74,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,9]]}},"alternative-id":["9781"],"URL":"https:\/\/doi.org\/10.1007\/s11219-026-09781-4","relation":{},"ISSN":["0963-9314","1573-1367"],"issn-type":[{"value":"0963-9314","type":"print"},{"value":"1573-1367","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,31]]},"assertion":[{"value":"5 December 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 August 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 August 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"All the authors of the manuscript declare no competing interests.","order":1,"name":"Ethics","label":"Competing Interests","group":{"name":"EthicsHeading","label":"Declarations"}}],"article-number":"49"}}