{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T11:54:53Z","timestamp":1791460493391,"version":"4.3.4"},"reference-count":59,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,4,26]],"date-time":"2023-04-26T00:00:00Z","timestamp":1682467200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"KKS Foundation"},{"DOI":"10.13039\/501100006359","name":"Blekinge Institute of Technology","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100006359","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Swedish Scientific Council","award":["2015-04913"],"award-info":[{"award-number":["2015-04913"]}]},{"name":"Basing Software Testing on Information Theory","award":["2020-05272"],"award-info":[{"award-number":["2020-05272"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2023,7,31]]},"abstract":"<jats:p>Non-robust (fragile) test execution is a commonly reported challenge in GUI-based test automation, despite much research and several proposed solutions. A test script needs to be resilient to (minor) changes in the tested application but, at the same time, fail when detecting potential issues that require investigation. Test script fragility is a multi-faceted problem. However, one crucial challenge is how to reliably identify and locate the correct target web elements when the website evolves between releases or otherwise fail and report an issue. This article proposes and evaluates a novel approach called similarity-based web element localization (Similo), which leverages information from multiple web element locator parameters to identify a target element using a weighted similarity score. This experimental study compares Similo to a baseline approach for web element localization. To get an extensive empirical basis, we target 48 of the most popular websites on the Internet in our evaluation. Robustness is considered by counting the number of web elements found in a recent website version compared to how many of these existed in an older version. Results of the experiment show that Similo outperforms the baseline; it failed to locate the correct target web element in 91 out of 801 considered cases (i.e., 11%) compared to 214 failed cases (i.e., 27%) for the baseline approach. The time efficiency of Similo was also considered, where the average time to locate a web element was determined to be 4 milliseconds. However, since the cost of web interactions (e.g., a click) is typically on the order of hundreds of milliseconds, the additional computational demands of Similo can be considered negligible. This study presents evidence that quantifying the similarity between multiple attributes of web elements when trying to locate them, as in our proposed Similo approach, is beneficial. With acceptable efficiency, Similo gives significantly higher effectiveness (i.e., robustness) than the baseline web element localization approach.<\/jats:p>","DOI":"10.1145\/3571855","type":"journal-article","created":{"date-parts":[[2022,11,22]],"date-time":"2022-11-22T11:58:41Z","timestamp":1669118321000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":46,"title":["Similarity-based Web Element Localization for Robust Test Automation"],"prefix":"10.1145","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8569-2290","authenticated-orcid":false,"given":"Michel","family":"Nass","sequence":"first","affiliation":[{"name":"SERL, Blekinge Institute of Technology, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7526-3727","authenticated-orcid":false,"given":"Emil","family":"Al\u00e9groth","sequence":"additional","affiliation":[{"name":"SERL, Blekinge Institute of Technology, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5179-4205","authenticated-orcid":false,"given":"Robert","family":"Feldt","sequence":"additional","affiliation":[{"name":"SERL, Blekinge Institute of Technology and Chalmers University of Technology, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5267-0602","authenticated-orcid":false,"given":"Maurizio","family":"Leotta","sequence":"additional","affiliation":[{"name":"DIBRIS, Universit\u00e0 di Genova, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3928-5408","authenticated-orcid":false,"given":"Filippo","family":"Ricca","sequence":"additional","affiliation":[{"name":"DIBRIS, Universit\u00e0 di Genova, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,4,26]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-011-9135-x"},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1145\/3102113.3102124","volume-title":"Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems","author":"Aldalur Inigo","year":"2017","unstructured":"Inigo Aldalur and Oscar Diaz. 2017. Addressing web locator fragility: A case for browser extensions. In Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems. 45\u201350."},{"key":"e_1_3_2_4_2","volume-title":"Visual GUI Testing: Automating High-level Software Testing in Industrial Practice","author":"Al\u00e9groth Emil","year":"2015","unstructured":"Emil Al\u00e9groth. 2015. Visual GUI Testing: Automating High-level Software Testing in Industrial Practice. Chalmers University of Technology."},{"key":"e_1_3_2_5_2","first-page":"1","volume-title":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST\u201915)","author":"Alegroth Emil","year":"2015","unstructured":"Emil Alegroth, Geoffrey Bache, and Emily Bache. 2015. On the industrial applicability of TextTest: An empirical case study. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST\u201915). IEEE, 1\u201310."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-016-9497-6"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2013.14"},{"key":"e_1_3_2_8_2","first-page":"172","volume-title":"2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST\u201918)","author":"Al\u00e9groth Emil","year":"2018","unstructured":"Emil Al\u00e9groth, Arvid Karlsson, and Alexander Radway. 2018. Continuous integration and visual GUI testing: Benefits and drawbacks in industrial practice. In 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST\u201918). IEEE, 172\u2013181."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2018.08.007"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/2610384.2610391"},{"key":"e_1_3_2_11_2","first-page":"4","volume-title":"Proceedings of the 8th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement","author":"Bauersfeld Sebastian","year":"2014","unstructured":"Sebastian Bauersfeld, Tanja E. J. Vos, Nelly Condori-Fernandez, Alessandra Bagnato, and Etienne Brosse. 2014. Evaluating the TESTAR tool in an industrial case study. In Proceedings of the 8th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement. ACM, 4."},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2021.106754"},{"key":"e_1_3_2_13_2","first-page":"24","volume-title":"Proceedings of the 1st International Workshop on End-to-end Test Script Engineering","author":"Choudhary Shauvik Roy","year":"2011","unstructured":"Shauvik Roy Choudhary, Dan Zhao, Husayn Versee, and Alessandro Orso. 2011. Water: Web application test repair. In Proceedings of the 1st International Workshop on End-to-end Test Script Engineering. 24\u201329."},{"key":"e_1_3_2_14_2","first-page":"354","volume-title":"2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST\u201918)","author":"Debroy Vidroha","year":"2018","unstructured":"Vidroha Debroy, Lance Brimble, Matthew Yost, and Archana Erry. 2018. Automating web application testing from the ground up: Experiences and lessons learned in an industrial setting. In 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST\u201918). IEEE, 354\u2013362."},{"key":"e_1_3_2_15_2","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1109\/ISSRE.2019.00035","volume-title":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE\u201919)","author":"Dobslaw Felix","year":"2019","unstructured":"Felix Dobslaw, Robert Feldt, David Micha\u00eblsson, Patrik Haar, Francisco Gomes de Oliveira Neto, and Richard Torkar. 2019. Estimating return on investment for GUI test automation frameworks. In 2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE\u201919). IEEE, 271\u2013282."},{"key":"e_1_3_2_16_2","first-page":"327","volume-title":"2018 13th International Conference on Computer Engineering and Systems (ICCES\u201918)","author":"Eladawy Hadeel Mohamed","year":"2018","unstructured":"Hadeel Mohamed Eladawy, Amr E. Mohamed, and Sameh A. Salem. 2018. A new algorithm for repairing web-locators using optimization techniques. In 2018 13th International Conference on Computer Engineering and Systems (ICCES\u201918). IEEE, 327\u2013331."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3121245.3121250"},{"key":"e_1_3_2_18_2","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1109\/QRS.2018.00034","volume-title":"2018 IEEE International Conference on Software Quality, Reliability and Security (QRS\u201918)","author":"Grechanik Mark","year":"2018","unstructured":"Mark Grechanik, Chi Wu Mao, Ankush Baisal, David Rosenblum, and B. M. Mainul Hossain. 2018. Differencing graphical user interfaces. In 2018 IEEE International Conference on Software Quality, Reliability and Security (QRS\u201918). IEEE, 203\u2013214."},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1109\/ICSTW.2009.31","volume-title":"International Conference on Software Testing, Verification and Validation Workshops, 2009 (ICSTW\u201909).","author":"Grechanik Mark","year":"2009","unstructured":"Mark Grechanik, Qing Xie, and Chen Fu. 2009. Creating GUI testing tools using accessibility technologies. In International Conference on Software Testing, Verification and Validation Workshops, 2009 (ICSTW\u201909). IEEE, 243\u2013250."},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1109\/ICSM.2009.5306345","volume-title":"2009 IEEE International Conference on Software Maintenance","author":"Grechanik Mark","year":"2009","unstructured":"Mark Grechanik, Qing Xie, and Chen Fu. 2009. Experimental assessment of manual versus tool-based maintenance of GUI-directed test scripts. In 2009 IEEE International Conference on Software Maintenance. IEEE, 9\u201318."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2009.5070540"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2950294"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2010.36"},{"key":"e_1_3_2_24_2","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1109\/ICDH.2018.00035","volume-title":"2018 7th International Conference on Digital Home (ICDH\u201918)","author":"Jiang Weina","year":"2018","unstructured":"Weina Jiang, Xiaozhe Li, and Xinming Wang. 2018. A black-box based script repair method for GUI regression test. In 2018 7th International Conference on Digital Home (ICDH\u201918). IEEE, 148\u2013153."},{"issue":"4","key":"e_1_3_2_25_2","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1109\/SANER.2019.8667976","article-title":"COLOR: Correct locator recommender for broken test scripts using various clues in web application","volume":"36","author":"Kirinuki Hiroyuki","year":"2019","unstructured":"Hiroyuki Kirinuki, Haruto Tanno, and Katsuyuki Natsukawa. 2019. COLOR: Correct locator recommender for broken test scripts using various clues in web application. 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER) 36, 4 (2019), 310\u2013320.","journal-title":"2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER)"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/2994291.2994299"},{"key":"e_1_3_2_27_2","first-page":"1","volume-title":"2013 8th Iberian Conference on Information Systems and Technologies (CISTI\u201913)","author":"La\u0163iu Gen\u0163iana Ioana","year":"2013","unstructured":"Gen\u0163iana Ioana La\u0163iu, Octavian Cre\u0163, and Lucia V\u0103cariu. 2013. Graphical user interface testing using evolutionary algorithms. In 2013 8th Iberian Conference on Information Systems and Technologies (CISTI\u201913). IEEE, 1\u20136."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/2489280.2489284"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1767"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSREW.2014.17"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2015.7102611"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1002\/smr.1771"},{"key":"e_1_3_2_33_2","first-page":"161","volume-title":"2017 IEEE International Conference on Software Testing, Verification and Validation (ICST\u201917)","author":"Li Xiao","year":"2017","unstructured":"Xiao Li, Nana Chang, Yan Wang, Haohua Huang, Yu Pei, Linzhang Wang, and Xuandong Li. 2017. ATOM: Automatic maintenance of GUI test scripts for evolving mobile applications. In 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST\u201917). IEEE, 161\u2013171."},{"key":"e_1_3_2_34_2","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1109\/SEAA.2013.29","volume-title":"2013 39th Euromicro Conference on Software Engineering and Advanced Applications","author":"Liebel Grischa","year":"2013","unstructured":"Grischa Liebel, Emil Al\u00e9groth, and Robert Feldt. 2013. State-of-practice in GUI-based system and acceptance testing: An industrial multiple-case study. In 2013 39th Euromicro Conference on Software Engineering and Advanced Applications. IEEE, 17\u201324."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1484"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.908959"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.datak.2010.12.001"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1629"},{"key":"e_1_3_2_39_2","first-page":"364","volume-title":"2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW\u201920)","author":"Nass Michel","year":"2020","unstructured":"Michel Nass, Emil Al\u00e9groth, and Robert Feldt. 2020. On the industrial applicability of augmented testing: An empirical study. In 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW\u201920). IEEE, 364\u2013371."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2021.106625"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1760"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.5555\/948785.948830"},{"key":"e_1_3_2_43_2","unstructured":"Online. 2022. Alexa . https:\/\/www.alexa.com\/topsites\/countries\/US."},{"key":"e_1_3_2_44_2","unstructured":"Online. 2022. CSS Selectors . https:\/\/en.wikipedia.org\/wiki\/CSS."},{"key":"e_1_3_2_45_2","unstructured":"Online. 2022. DOM . https:\/\/www.w3.org\/TR\/WD-DOM\/introduction.html."},{"key":"e_1_3_2_46_2","unstructured":"Online. 2022. GitHub . https:\/\/github.com\/SeleniumHQ\/selenium-ide."},{"key":"e_1_3_2_47_2","unstructured":"Online. 2022. Internet Archive . https:\/\/web.archive.org\/."},{"key":"e_1_3_2_48_2","unstructured":"Online. 2022. Levenshtein . http:\/\/levenshtein.net."},{"key":"e_1_3_2_49_2","unstructured":"Online. 2022. Replication Package . https:\/\/github.com\/michelnass\/Similo2."},{"key":"e_1_3_2_50_2","unstructured":"Online. 2022. Selenium . https:\/\/www.seleniumhq.org."},{"key":"e_1_3_2_51_2","unstructured":"Online. 2022. Selenium IDE . https:\/\/www.selenium.dev\/selenium-ide."},{"key":"e_1_3_2_52_2","unstructured":"Online. 2022. XPath . https:\/\/en.wikipedia.org\/wiki\/XPath."},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1016\/bs.adcom.2018.10.005"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.5555\/2486788.2486926"},{"key":"e_1_3_2_55_2","doi-asserted-by":"crossref","first-page":"881","DOI":"10.1109\/ICSE.2012.6227131","volume-title":"2012 34th International Conference on Software Engineering (ICSE\u201912)","author":"Thummalapenta Suresh","year":"2012","unstructured":"Suresh Thummalapenta, Saurabh Sinha, Nimit Singhania, and Satish Chandra. 2012. Automating test automation. In 2012 34th International Conference on Software Engineering (ICSE\u201912). IEEE, 881\u2013891."},{"key":"e_1_3_2_56_2","first-page":"1","volume-title":"Advances in Computers","author":"Tonella Paolo","year":"2014","unstructured":"Paolo Tonella, Filippo Ricca, and Alessandro Marchetto. 2014. Recent advances in web testing. In Advances in Computers, Vol. 93. Elsevier, 1\u201351."},{"key":"e_1_3_2_57_2","first-page":"395","volume-title":"2009 IEEE International Conference on Software Maintenance","author":"Xie Qing","year":"2009","unstructured":"Qing Xie, Mark Grechanik, Chen Fu, and Chad Cumby. 2009. Guide: A GUI differentiator. In 2009 IEEE International Conference on Software Maintenance. IEEE, 395\u2013396."},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/2610384.2610390"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.1078"},{"key":"e_1_3_2_60_2","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1109\/QRS-C.2018.00041","volume-title":"2018 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C\u201918)","author":"Zheng Yu","year":"2018","unstructured":"Yu Zheng, Song Huang, Zhan-wei Hui, and Ya-Ning Wu. 2018. A method of optimizing multi-locators based on machine learning. In 2018 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C\u201918). IEEE, 172\u2013174."}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3571855","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3571855","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:48:48Z","timestamp":1750182528000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3571855"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,26]]},"references-count":59,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,7,31]]}},"alternative-id":["10.1145\/3571855"],"URL":"https:\/\/doi.org\/10.1145\/3571855","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,26]]},"assertion":[{"value":"2022-05-09","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-10-18","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-04-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}