{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T17:16:36Z","timestamp":1790961396701,"version":"4.1.0"},"reference-count":34,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","license":[{"start":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:00:00Z","timestamp":1750291200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-sa\/4.0\/legalcode"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62372227"],"award-info":[{"award-number":["62372227"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2025,6,19]]},"abstract":"<jats:p>Model-based testing (MBT) has been an important methodology in software engineering, attracting extensive research attention for over four decades. However, despite its academic acclaim, studies examining the impact of MBT in industrial environments\u2014particularly regarding its extended effects\u2014remain limited and yield unclear results. This gap may contribute to the challenges in establishing a study environment for implementing and applying MBT in production settings to evaluate its impact over time. To bridge this gap, we collaborated with an industrial partner to undertake a comprehensive, longitudinal empirical study employing mixed methods. Over two months, we implemented our MBT tool within the corporation, assessing the immediate and extended effectiveness and efficiency of MBT compared to script-writing-based testing. Through a mix of quantitative and qualitative methods\u2014spanning controlled experiments, questionnaire surveys, and interviews\u2014our study uncovers several insightful findings. These include differences in effectiveness and maintainability between immediate and extended MBT application, the evolving perceptions and expectations of engineers regarding MBT, and more. Leveraging these insights, we propose actionable implications for both the academic and industrial communities, aimed at bolstering confidence in MBT adoption and investment for software testing purposes.<\/jats:p>","DOI":"10.1145\/3715789","type":"journal-article","created":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T11:15:34Z","timestamp":1750331734000},"page":"1545-1566","source":"Crossref","is-referenced-by-count":2,"title":["A Mixed-Methods Study of Model-Based GUI Testing in Real-World Industrial Settings"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-6929-1027","authenticated-orcid":false,"given":"Shaoheng","family":"Cao","sequence":"first","affiliation":[{"name":"Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-4367-1228","authenticated-orcid":false,"given":"Renyi","family":"Chen","sequence":"additional","affiliation":[{"name":"Samsung Electronics (China) R&amp;D Centre, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1213-6733","authenticated-orcid":false,"given":"Wenhua","family":"Yang","sequence":"additional","affiliation":[{"name":"Nanjing University of Aeronautics and Astronautics, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4011-5350","authenticated-orcid":false,"given":"Minxue","family":"Pan","sequence":"additional","affiliation":[{"name":"Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3090-9568","authenticated-orcid":false,"given":"Xuandong","family":"Li","sequence":"additional","affiliation":[{"name":"Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,6,19]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-022-10145-2"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9570-9"},{"key":"e_1_3_1_4_2","first-page":"176","volume-title":"Mobile Web Information Systems","author":"Brambilla Marco","year":"2014","unstructured":"Marco Brambilla, Andrea Mauri, and Eric Umuhoza. 2014. Extending the Interaction Flow Modeling Language (IFML) for Model Driven Development of Mobile Applications Front End. In Mobile Web Information Systems, Irfan Awan, Muhammad Younas, Xavier Franch, and Carme Quer (Eds.). Springer International Publishing, Cham, 176\u2013191."},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3691620.3695032"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","unstructured":"Paul Clarke Rory V. O\u2019Connor and Brian Leavy. 2016. A Complexity Theory Viewpoint on the Software Development Process and Situational Context. In 2016 IEEE\/ACM International Conference on Software and System Processes (ICSSP). 86\u201390. https:\/\/doi.org\/10.1145\/2904354.2904369 10.1145\/2904354.2904369","DOI":"10.1145\/2904354.2904369"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","unstructured":"S.R. Dalal A. Jain N. Karunanithi J.M. Leaton C.M. Lott G.C. Patton and B.M. Horowitz. 1999. Model-based testing in practice. In Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002). 285\u2013294. https:\/\/doi.org\/10.1145\/302405.302640 10.1145\/302405.302640","DOI":"10.1145\/302405.302640"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.entcs.2015.05.002"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","unstructured":"Vladimir Entin Mathias Winder Bo Zhang and Stephan Christmann. 2011. Combining Model-Based and Capture-Replay Testing Techniques of Graphical User Interfaces: An Industrial Approach. In 2011 IEEE Fourth International Conference on Software Testing Verification and Validation Workshops. 572\u2013577. https:\/\/doi.org\/10.1109\/ICSTW.2011.13 10.1109\/ICSTW.2011.13","DOI":"10.1109\/ICSTW.2011.13"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","unstructured":"Vladimir Entin Mathias Winder Bo Zhang and Stephan Christmann. 2012. Introducing model-based testing in an industrial scrum project. In 2012 7th International Workshop on Automation of Software Test (AST). 43\u201349. https:\/\/doi.org\/10.1109\/IWAST.2012.6228989 10.1109\/IWAST.2012.6228989","DOI":"10.1109\/IWAST.2012.6228989"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2023.102942"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09803-0"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.111032"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","unstructured":"Robert Grissom and John Kim. 2005. Effect Sizes for Research: A Broad Practical Approach. https:\/\/doi.org\/10.4324\/9781410612915 10.4324\/9781410612915","DOI":"10.4324\/9781410612915"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","unstructured":"Katharina G\u00f6tz Patric Feldmeier and Gordon Fraser. 2022. Model-based Testing of Scratch Programs. In 2022 IEEE Conference on Software Testing Verification and Validation (ICST). 411\u2013421. https:\/\/doi.org\/10.1109\/ICST53961.2022.00047 10.1109\/ICST53961.2022.00047","DOI":"10.1109\/ICST53961.2022.00047"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","unstructured":"Stefan Karlsson Adnan \u010cau\u0161evi\u0107 Daniel Sundmark and M\u00e5rten Larsson. 2021. Model-based Automated Testing of Mobile Applications: An Industrial Case Study. In 2021 IEEE International Conference on Software Testing Verification and Validation Workshops (ICSTW). 130\u2013137. https:\/\/doi.org\/10.1109\/ICSTW52544.2021.00033 10.1109\/ICSTW52544.2021.00033","DOI":"10.1109\/ICSTW52544.2021.00033"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.csi.2017.08.002"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.5555\/3044751"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3623344"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1002\/casp.2450040506"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2023.103005"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510133"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","unstructured":"Arthur Marques Franklin Ramalho and Wilkerson L. Andrade. 2014. Comparing Model-Based Testing with Traditional Testing Strategies: An Empirical Study. In 2014 IEEE Seventh International Conference on Software Testing Verification and Validation Workshops. 264\u2013273. https:\/\/doi.org\/10.1109\/ICSTW.2014.29 10.1109\/ICSTW.2014.29","DOI":"10.1109\/ICSTW.2014.29"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-021-09575-w"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1038\/134201b0"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-021-00924-8"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.110433"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544902.3546233"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/1808266.1808279"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/2591062.2591180"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-016-9321-y"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","unstructured":"Tommi Takala Mika Katara and Julian Harty. 2011. Experiences of System-Level Model-Based GUI Testing of an Android Application. In 2011 Fourth IEEE International Conference on Software Testing Verification and Validation. 377\u2013386. https:\/\/doi.org\/10.1109\/ICST.2011.11 10.1109\/ICST.2011.11","DOI":"10.1109\/ICST.2011.11"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.5555\/1200168"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598033"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3452383.3452388"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3715789","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T10:57:32Z","timestamp":1787482652000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3715789"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,19]]},"references-count":34,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2025,6,19]]}},"alternative-id":["10.1145\/3715789"],"URL":"https:\/\/doi.org\/10.1145\/3715789","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,19]]}}}