{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T18:32:15Z","timestamp":1785436335614,"version":"3.56.0"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,11,24]],"date-time":"2022-11-24T00:00:00Z","timestamp":1669248000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,11,24]],"date-time":"2022-11-24T00:00:00Z","timestamp":1669248000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Univ Access Inf Soc"],"published-print":{"date-parts":[[2024,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The importance of guaranteeing accessible Web applications is becoming widely recognised, and supported by national and international legislation. This implies an increasing need for large scale validations, which can be achieved only through automatic support. Thus, there is a need for a new generation of accessibility validation tools able to check against the continuously evolving guidelines and report any issues revealed, considering that their results will be used by many people with varied backgrounds and goals. Such tools should be able to support monitoring of Web sites\u2019 accessibility, provide user-friendly information suitable for various purposes, and be transparent in terms of what they are actually able to evaluate in order to elicit the right expectations from their users. They should also consider how the technologies for implementing Web sites have evolved in recent years. In this paper, we provide a description of the requirements and design dimensions that characterise such tools in order to address emerging needs, providing indications on whether and how several existing validation tools support them, and present how we have addressed and implemented them in a specific tool. We also report on a first usability test of such tool, which has provided encouraging feedback.<\/jats:p>","DOI":"10.1007\/s10209-022-00948-x","type":"journal-article","created":{"date-parts":[[2022,11,24]],"date-time":"2022-11-24T20:49:04Z","timestamp":1669322944000},"page":"435-454","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Usability and transparency in the design of a tool for automatic support for web accessibility validation"],"prefix":"10.1007","volume":"23","author":[{"given":"Nicola","family":"Iannuzzi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marco","family":"Manca","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8355-6909","authenticated-orcid":false,"given":"Fabio","family":"Patern\u00f2","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carmen","family":"Santoro","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,11,24]]},"reference":[{"issue":"3","key":"948_CR1","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1145\/1744161.1744168","volume":"17","author":"J Gulliksen","year":"2010","unstructured":"Gulliksen, J., Von Axelson, H., Persson, H., G\u00f6ransson, H.: Accessibility and public policy in Sweden. Interactions 17(3), 26\u201329 (2010)","journal-title":"Interactions"},{"key":"948_CR2","doi-asserted-by":"crossref","unstructured":"Lazar, J., & Olalere, A. (2011). Investigation of best practices for maintaining section 508 Compliance in US federal Web sites.In: International conference on universal access in human-computer interaction pp. 498\u2013506. Berlin: Springer.","DOI":"10.1007\/978-3-642-21672-5_55"},{"issue":"3","key":"948_CR3","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1145\/2745395","volume":"22","author":"F Patern\u00f2","year":"2015","unstructured":"Patern\u00f2, F., Schiavone, A.: The role of tool support in public policies and accessibility. ACM Interact 22(3), 60\u201363 (2015)","journal-title":"ACM Interact"},{"key":"948_CR4","unstructured":"EU Commission. (2016, October 26). Directive (EU) 2016\/2102 of the European Parliament and of the Council. Retrieved from https:\/\/eur-lex.europa.eu: https:\/\/eur-lex.europa.eu\/eli\/dir\/2016\/2102\/oj"},{"key":"948_CR5","doi-asserted-by":"crossref","unstructured":"Power, C., Freire, A., Petrie, H., & Swallow, D. (2012). Guidelines are only half of the story: accessibility problems encountered by blind users on the Web. In: Proceedings of the SIGCHI conference on human factors in computing systems pp. 433\u2013442. ACM.","DOI":"10.1145\/2207676.2207736"},{"key":"948_CR6","first-page":"362","volume":"2545","author":"A Beirekdar","year":"2002","unstructured":"Beirekdar, A., Vanderdonckt, J., Noirhomme-Fraiture, M.: Kwaresmi\u2013Knowledge-based Web Automated Evaluation with REconfigurable guidelineS optimisation. (Springer, Ed) DSV-IS 2545, 362\u2013376 (2002)","journal-title":"(Springer, Ed) DSV-IS"},{"key":"948_CR7","doi-asserted-by":"crossref","unstructured":"Beirekdar A., Keita M., Noirhomme M., Randolet F., Vanderdonckt J., Mariage C. (2005) Flexible Reporting for Automated Usability and Accessibility Evaluation of Web Sites. In: Costabile M.F., Patern\u00f2 F (eds) Human-Computer Interaction - INTERACT 2005, INTERACT 2005, Lecture Notes in Computer Science, Springer, Berlin, Heidelberg vol. 3585.","DOI":"10.1007\/11555261_25"},{"key":"948_CR8","doi-asserted-by":"crossref","unstructured":"Fernandes, N., Kaklanis, N., Votis, K., Tzovaras, D., & Carri\u00e7o, L. (2014). An analysis of spersonalised Web accessibility. In: Proceedings of the 11th web for all conference pp. 19. ACM.","DOI":"10.1145\/2596695.2596698"},{"key":"948_CR9","doi-asserted-by":"crossref","unstructured":"Fuertes, J. L., Gonz\u00e1lez, R., Guti\u00e9rrez, E., & Mart\u00ednez, L. (2009). Hera-FFX: a Firefox add-on for semi-automatic Web accessibility evaluation. In: Proceedings of the 2009 International cross-disciplinary conference on web accessibility (W4A) pp. 26\u201335. ACM.","DOI":"10.1145\/1535654.1535661"},{"key":"948_CR10","first-page":"117","volume":"2003","author":"MY Ivory","year":"2003","unstructured":"Ivory, M.Y., Mankoff, J., Le, A.: Using automated tools to improve Web site usage by users with diverse abilities. Hum-Comput Interact Inst 2003, 117 (2003)","journal-title":"Hum-Comput Interact Inst"},{"key":"948_CR11","unstructured":"Nietzio, A., Eibegger, M., Goodwin, M., & Snaprud, M. (2011). Towards a score function for WCAG 2.0 benchmarking.In: Proceedings of W3C online symposium on website accessibility metrics. Retrieved from https:\/\/www.w3.org\/WAI\/RD\/2011\/metrics\/paper11"},{"issue":"1","key":"948_CR12","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1007\/s10209-014-0399-3","volume":"14","author":"A Schiavone","year":"2015","unstructured":"Schiavone, A., Patern\u00f2, F.: An extensible environment for guideline-based accessibility evaluation of dynamic Web applications. Univers Access Inf Soc, Springer Verlag 14(1), 111\u2013132 (2015)","journal-title":"Univers Access Inf Soc, Springer Verlag"},{"key":"948_CR13","doi-asserted-by":"crossref","unstructured":"Gay, G, and Qi Li, C. 2010. AChecker: open, interactive, customisable, Web accessibility checking. In: Proceedings of the 2010 International cross disciplinary conference on web accessibility (W4A). 1\u20132.","DOI":"10.1145\/1805986.1806019"},{"key":"948_CR14","doi-asserted-by":"crossref","unstructured":"Mirri, S., Muratori, L. A., & Salomoni, P. (2011). Monitoring accessibility: large scale evaluations at a geo political level. In: The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility pp. 163\u2013170. New York: ACM.","DOI":"10.1145\/2049536.2049566"},{"key":"948_CR15","doi-asserted-by":"publisher","unstructured":"Miniukovich, A., Scaltritti, M., Sulpizio, S., and De Angeli, A. 2019. Guideline-Based Evaluation of Web Readability. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI \u201919). Association for Computing Machinery, New York, NY, USA, Paper 508, pp. 1\u201312. DOI:https:\/\/doi.org\/10.1145\/3290605.3300738","DOI":"10.1145\/3290605.3300738"},{"key":"948_CR16","doi-asserted-by":"publisher","unstructured":"Moreno, L., Alarcon, R., Segura-Bedmar, I., and Mart\u00ednez, P. 2019. Lexical simplification approach to support the accessibility guidelines. In: Proceedings of the XX international conference on human computer interaction (interaccion \u201919). association for computing machinery, New York, NY, USA, Article 14, pp. 1\u20134. DOI:https:\/\/doi.org\/10.1145\/3335595.3335651","DOI":"10.1145\/3335595.3335651"},{"key":"948_CR17","doi-asserted-by":"publisher","first-page":"479","DOI":"10.1007\/978-1-4471-7440-0_26","volume-title":"Tools for Web accessibility evaluation","author":"J Abascal","year":"2019","unstructured":"Abascal, J., Argue, M., Valencia, X.: Tools for Web accessibility evaluation, pp. 479\u2013503. Springer, Web Accessibility, London (2019)"},{"key":"948_CR18","volume-title":"The Transparency of Automatic Web Accessibility Evaluation Tools: Design Criteria, State of the Art, and User Perception","author":"M Manca","year":"2022","unstructured":"Manca, M., Palumbo, V., Patern\u00f2, F., Santoro, C.: The Transparency of Automatic Web Accessibility Evaluation Tools: Design Criteria, State of the Art, and User Perception. ACM Press, ACM Transaction on Accessible Computing (2022)"},{"issue":"1\u201383","key":"948_CR19","first-page":"24","volume":"83","author":"B Broccia","year":"2020","unstructured":"Broccia, B., Manca, M., Patern\u00f2, F., Pulina, F.: Flexible Automatic Support for Web Accessibility Validation. Proc ACM Hum Comput Interact EICS 83(1\u201383), 24 (2020)","journal-title":"Proc ACM Hum Comput Interact EICS"},{"key":"948_CR20","doi-asserted-by":"publisher","DOI":"10.3389\/fcomp.2021.605772","volume":"3","author":"J Pelzetter","year":"2021","unstructured":"Pelzetter, J.: A Declarative Model for Web Accessibility Requirements and its Implementation. Frontiers Comput. Sci. 3, 605772 (2021)","journal-title":"Frontiers Comput. Sci."},{"key":"948_CR21","doi-asserted-by":"crossref","unstructured":"Vigo, M., Brown, J., and Conway, V. 2013. Benchmarking Web accessibility evaluation tools: measuring the harm of sole reliance on automated tests. In: Proceedings of the 10th international cross-disciplinary conference on web accessibility. pp. 1\u201310.","DOI":"10.1145\/2461121.2461124"},{"issue":"2021","key":"948_CR22","first-page":"2","volume":"3","author":"A Burkard","year":"2021","unstructured":"Burkard, A., Zimmermann, G., Schwarzer, B.: Monitoring systems for checking websites on accessibility. Front Comput Sci 3(2021), 2 (2021)","journal-title":"Front Comput Sci"},{"key":"948_CR23","doi-asserted-by":"crossref","unstructured":"Abduganiev S.G.: Towards automated web\u00a0accessibility evaluation: a comparative study. Int J Inf Technol\u00a0Comput Sci (IJITCS) 9(9), 18\u201344 (2017)","DOI":"10.5815\/ijitcs.2017.09.03"},{"key":"948_CR24","unstructured":"P\u0103dure, M., and Pribeanu, C. Exploring the differences between five accessibility evaluation tools (2019)"},{"key":"948_CR25","doi-asserted-by":"publisher","unstructured":"Fraz\u00e3o, T., and Duarte, C. Comparing accessibility evaluation plug-ins. In: Proceedings of the 17th International web for all conference (W4A '20). association for computing machinery, New York, NY, USA, Article 20, 1\u201311 (2020). https:\/\/doi.org\/10.1145\/3371300.3383346","DOI":"10.1145\/3371300.3383346"},{"issue":"2","key":"948_CR26","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1080\/0144929X.2013.848238","volume":"34","author":"Y Yesilada","year":"2015","unstructured":"Yesilada, Y., Brajnik, G., Vigo, M., Harper, S.: Exploring perceptions of Web accessibility: a survey approach. Behav Inf Technol 34(2), 119\u2013134 (2015)","journal-title":"Behav Inf Technol"},{"issue":"2","key":"948_CR27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2141943.2141946","volume":"4","author":"G Brajnik","year":"2012","unstructured":"Brajnik, G., Yesilada, Y., Harper, S.: Is accessibility conformance an elusive property? A study of validity and reliability of WCAG 2.0. ACM Trans Access Comput (TACCESS) 4(2), 1\u201328 (2012)","journal-title":"ACM Trans Access Comput (TACCESS)"},{"issue":"2","key":"948_CR28","first-page":"218","volume":"7","author":"AM Molinero","year":"2006","unstructured":"Molinero, A.M., Kohun, F.G., Morris, R.: Reliability in Automated evaluation tools for web accessibility standards compliance. Issues Inf Syst 7(2), 218\u2013222 (2006)","journal-title":"Issues Inf Syst"},{"key":"948_CR29","doi-asserted-by":"publisher","unstructured":"Petrie, H., King, N., Velasco, C., Gappa, H., Nordbrock, G. (2007): The usability of accessibility evaluation tools, In: Stephanidis, C. ed UAHCI 2007, LNCS, vol. 4556, pp. 124\u2013132. Springer Heidelberg, https:\/\/doi.org\/10.1007\/978-3-540-73283-9_15","DOI":"10.1007\/978-3-540-73283-9_15"},{"key":"948_CR30","doi-asserted-by":"publisher","unstructured":"Salehnamadi, N., Alshayban, A., Lin, JW, Ahmed, I., Branham, S., and Malek, S. 2021. Latte: Use-Case and Assistive-Service Driven Automated Accessibility Testing Framework for Android. In: CHI Conference on Human Factors in Computing Systems (CHI \u201921), May 8\u2013 13, 2021, Yokohama, Japan. ACM, New York, NY, USA, 11 pages. https:\/\/doi.org\/10.1145\/3411764.3445455","DOI":"10.1145\/3411764.3445455"},{"key":"948_CR31","doi-asserted-by":"publisher","unstructured":"Patern\u00f2 F., Pulina F., Santoro C., Gappa H., Mohamad Y. (2020) Requirements for Large Scale Web Accessibility Evaluation. In: Miesenberger K., Manduchi R., Covarrubias Rodriguez M., Pe\u0148\u00e1z P. (eds) Computers Helping People with Special Needs. ICCHP 2020. Lecture Notes in Computer Science, vol. 12376. Springer, Cham. https:\/\/doi.org\/10.1007\/978-3-030-58796-3_33","DOI":"10.1007\/978-3-030-58796-3_33"},{"key":"948_CR32","unstructured":"Siteimprove, QualWeb. https:\/\/chrome.google.com\/webstore\/detail\/siteimprove-accessibility\/djcglbmbegflehmbfleechkjhmedcopn, http:\/\/qualweb.di.fc.ul.pt\/evaluator\/. Accessed 6 Sept 2022"},{"key":"948_CR33","unstructured":"Achecker, https:\/\/achecker.achecks.ca\/checker\/index.php. Accessed 6 Sept 2022"},{"key":"948_CR34","unstructured":"Experte, https:\/\/www.experte.com\/accessibility. Accessed 6 Sept 2022"},{"key":"948_CR35","unstructured":"IBM Accessibility Checker, https:\/\/addons.mozilla.org\/en-US\/firefox\/addon\/accessibility-checker\/. Accessed 6 Sept 2022"},{"key":"948_CR36","unstructured":"Lighthouse, https:\/\/web.dev\/lighthouse-accessibility\/. Accessed 6 Sept 2022"},{"key":"948_CR37","unstructured":"Shadi Abou-Zahra. Evaluation and Report Language (EARL) (2017). Retrieved February 2, 2017 from https:\/\/www.w3.org\/TR\/EARL10-Schema\/#OutcomeValue"}],"container-title":["Universal Access in the Information Society"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10209-022-00948-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10209-022-00948-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10209-022-00948-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,13]],"date-time":"2024-03-13T21:06:05Z","timestamp":1710363965000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10209-022-00948-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,24]]},"references-count":37,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,3]]}},"alternative-id":["948"],"URL":"https:\/\/doi.org\/10.1007\/s10209-022-00948-x","relation":{},"ISSN":["1615-5289","1615-5297"],"issn-type":[{"value":"1615-5289","type":"print"},{"value":"1615-5297","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,24]]},"assertion":[{"value":"8 November 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 November 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}