{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T11:40:13Z","timestamp":1764934813381,"version":"3.46.0"},"reference-count":21,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2025,2,15]],"date-time":"2025-02-15T00:00:00Z","timestamp":1739577600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,2,15]],"date-time":"2025-02-15T00:00:00Z","timestamp":1739577600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"ISTI - PISA"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Univ Access Inf Soc"],"published-print":{"date-parts":[[2025,8]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Accessibility validation of online digital content through automatic tools has been addressed in limited terms. There is a need for a holistic approach to accessibility validation, able to align well with the goals of public organisations and accessibility authorities. In this perspective, one main issue is that usually such tools aim to assess either the web content, or the PDF documents, but not both. However, users need that all the content be accessible, regardless of the format used, thus it would be helpful to have tools able to perform a combined analysis of both web pages, and PDF files connected with such pages. This combined approach to accessibility validation, beyond providing a more complete and coherent view on the accessibility supported for everyone, would also be important for accessibility authorities (who must monitor the state of accessibility on a large scale, e.g., to comply with the EU WAD Directive), as well as for public organisations (since they could incur in legal risks and potential lawsuits, if only web pages are compliant). Indeed, a combined accessibility analysis would help them to identify the sites that need more interventions and would also be useful more generally to stimulate web developers and content providers to pay attention to both aspects. In this paper, we present how a tool that originally supported Web accessibility validation only, has been extended to include in the supported accessibility analysis also PDF files, to provide a more comprehensive assessment of the evaluated content. We describe the tasks it supports, the possible use cases, as well as some results obtained in a large-scale combined validation (Web and PDF content), carried out on Italian public service websites. The paper also reports the results of a user study carried out to understand the usability of the features that were added to the tool to support the validation of the accessibility of both web and PDF content.<\/jats:p>","DOI":"10.1007\/s10209-025-01194-7","type":"journal-article","created":{"date-parts":[[2025,2,14]],"date-time":"2025-02-14T23:24:58Z","timestamp":1739575498000},"page":"2315-2334","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Combined accessibility validation and monitoring of web sites and PDF documents"],"prefix":"10.1007","volume":"24","author":[{"given":"Nicola","family":"Iannuzzi","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marco","family":"Manca","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fabio","family":"Patern\u00f2","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Carmen","family":"Santoro","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,2,15]]},"reference":[{"key":"1194_CR1","doi-asserted-by":"publisher","unstructured":"Acosta-Vargas, P., Luj\u00e1n-Mora, S., and Acosta, T.: Accessibility of portable document format in education repositories. In: Proceedings of the 2017 9th International Conference on Education Technology and Computers, December 20, 2017. ACM, Barcelona Spain. pp. 239\u2013242 (2017). https:\/\/doi.org\/10.1145\/3175536.3175574","DOI":"10.1145\/3175536.3175574"},{"key":"1194_CR2","unstructured":"Apache.: Apache PDFBox, A Java PDF Library. Retrieved from https:\/\/pdfbox.apache.org\/"},{"issue":"EICS","key":"1194_CR3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3397871","volume":"4","author":"G Broccia","year":"2020","unstructured":"Broccia, G., Manca, M., Patern\u00f2, F., Pulina, F.: Flexible automatic support for web accessibility validation. Proc. ACM Hum.-Comput. Interact. 4(EICS), 1\u201324 (2020). https:\/\/doi.org\/10.1145\/3397871","journal-title":"Proc. ACM Hum.-Comput. Interact."},{"key":"1194_CR4","doi-asserted-by":"publisher","unstructured":"Carvalho, L.P., Guerreiro, T., Lawson, S., and Montague, K.: Towards real-time and large-scale web accessbility. In: The 25th International ACM SIGACCESS Conference on Computers and Accessibility, October 22, 2023, pp. 1\u20139. ACM, New York (2023). https:\/\/doi.org\/10.1145\/3597638.3608403","DOI":"10.1145\/3597638.3608403"},{"key":"1194_CR5","doi-asserted-by":"publisher","first-page":"113","DOI":"10.1007\/978-3-319-94277-3_20","volume-title":"Computers Helping People with Special Needs: 16th International Conference, ICCHP 2018, Linz, Austria, July 11-13, 2018, Proceedings, Part I","author":"A Darvishy","year":"2018","unstructured":"Darvishy, A.: PDF accessibility: tools and challenges. In: Miesenberger, K., Kouroupetroglou, G. (eds.) Computers Helping People with Special Needs: 16th International Conference, ICCHP 2018, Linz, Austria, July 11-13, 2018, Proceedings, Part I, pp. 113\u2013116. Springer, Cham (2018). https:\/\/doi.org\/10.1007\/978-3-319-94277-3_20"},{"issue":"2","key":"1194_CR6","doi-asserted-by":"publisher","first-page":"162","DOI":"10.17411\/jacces.v13i2.485","volume":"13","author":"M Debevc","year":"2023","unstructured":"Debevc, M., \u0160kraba, T., Cerovac, B., Ko\u017euh, I., Rajh, N.: Monitoring website accessibility: evaluating current approaches and a proposal for improvements. J. Accessib. Design All 13(2), 162\u2013187 (2023). https:\/\/doi.org\/10.17411\/jacces.v13i2.485","journal-title":"J. Accessib. Design All"},{"key":"1194_CR7","unstructured":"EU Commission.: Directive (EU) 2016\/2102 of the European Parliament and of the Council. Retrieved from https:\/\/eur-lex.europa.eu\/eli\/dir\/2016\/2102\/oj (2016)"},{"issue":"2","key":"1194_CR8","doi-asserted-by":"publisher","first-page":"188","DOI":"10.17411\/jacces.v13i2.421","volume":"13","author":"K Fernandes","year":"2023","unstructured":"Fernandes, K., Paramananthan, S., Cockburn, L., Nganji, J.: Readily available but how accessible?: An analysis of the web accessibility of healthcare-related resources. J. Access. Design All 13(2), 188\u2013215 (2023). https:\/\/doi.org\/10.17411\/jacces.v13i2.421","journal-title":"J. Access. Design All"},{"issue":"1","key":"1194_CR9","doi-asserted-by":"publisher","first-page":"435","DOI":"10.1007\/s10209-022-00948-x","volume":"23","author":"N Iannuzzi","year":"2022","unstructured":"Iannuzzi, N., Manca, M., Patern\u00f2, F., Santoro, C.: Usability and transparency in the design of a tool for automatic support for web accessibility validation. Univ. Access Inform. Soc. 23(1), 435\u2013454 (2022). https:\/\/doi.org\/10.1007\/s10209-022-00948-x","journal-title":"Univ. Access Inform. Soc."},{"key":"1194_CR10","doi-asserted-by":"publisher","unstructured":"Iannuzzi, N., Manca, M., Patern\u00f2, F., and Santoro, C.: Large scale automatic web accessibility validation. In: Proceedings of the 2023 ACM Conference on Information Technology for Social Good, September 06, 2023, pp. 307\u2013314. ACM, Lisbon Portugal (2023). https:\/\/doi.org\/10.1145\/3582515.3609549","DOI":"10.1145\/3582515.3609549"},{"key":"1194_CR11","doi-asserted-by":"publisher","DOI":"10.1201\/9781498710411","volume-title":"Usability Evaluation In Industry","year":"1996","unstructured":"Jordan, P.W., Thomas, B., McClelland, I.L., Weerdmeester, B. (eds.): Usability Evaluation In Industry. CRC Press, Boca Raton (1996). https:\/\/doi.org\/10.1201\/9781498710411"},{"issue":"1","key":"1194_CR12","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1007\/s10209-022-00956-x","volume":"23","author":"B Martins","year":"2022","unstructured":"Martins, B., Duarte, C.: Large-scale study of web accessibility metrics. Univ. Access Inf. Soc. 23(1), 411\u2013434 (2022). https:\/\/doi.org\/10.1007\/s10209-022-00956-x","journal-title":"Univ. Access Inf. Soc."},{"key":"1194_CR13","doi-asserted-by":"publisher","DOI":"10.1007\/s10209-023-01010-0","author":"B Martins","year":"2023","unstructured":"Martins, B., Duarte, C.: A large-scale web accessibility analysis considering technology adoption. Univ. Access Inf. Soc. (2023). https:\/\/doi.org\/10.1007\/s10209-023-01010-0","journal-title":"Univ. Access Inf. Soc."},{"issue":"3","key":"1194_CR14","doi-asserted-by":"publisher","first-page":"254","DOI":"10.1016\/j.lisr.2015.02.002","volume":"37","author":"JT Nganji","year":"2015","unstructured":"Nganji, J.T.: The portable document format (PDF) accessibility practice of four journal publishers. Lib. Inf. Sci. Res. 37(3), 254\u2013262 (2015). https:\/\/doi.org\/10.1016\/j.lisr.2015.02.002","journal-title":"Lib. Inf. Sci. Res."},{"key":"1194_CR15","doi-asserted-by":"publisher","unstructured":"Pandey, M. and Dong, T.: Blending accessibility in UI framework documentation to build awareness. In: The 25th International ACM SIGACCESS Conference on Computers and Accessibility, October 22, 2023, pp. 1\u201312. ACM, New York (2023). https:\/\/doi.org\/10.1145\/3597638.3608380","DOI":"10.1145\/3597638.3608380"},{"issue":"1","key":"1194_CR16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3507661","volume":"15","author":"D Pradhan","year":"2022","unstructured":"Pradhan, D., Rajput, T., Rajkumar, A.J., Lazar, J., Jain, R., Morariu, V.I., Manjunatha, V.: Development and evaluation of a tool for assisting content creators in making PDF files more accessible. ACM Trans. Access. Comput. 15(1), 1\u201352 (2022). https:\/\/doi.org\/10.1145\/3507661","journal-title":"ACM Trans. Access. Comput."},{"key":"1194_CR17","doi-asserted-by":"publisher","unstructured":"Rajkumar, A.J., Lazar, J., Jordan, J.B., Darvishy, A., and Hutter, H.P.: PDF accessibility of research papers: What tools are needed for assessment and remediation? (2020). https:\/\/doi.org\/10.24251\/HICSS.2020.512","DOI":"10.24251\/HICSS.2020.512"},{"issue":"1","key":"1194_CR18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3348797","volume":"13","author":"AS Ross","year":"2020","unstructured":"Ross, A.S., Zhang, X., Fogarty, J., Wobbrock, J.O.: An epidemiology-inspired large-scale analysis of android app accessibility. ACM Trans. Access. Comput. 13(1), 1\u201336 (2020). https:\/\/doi.org\/10.1145\/3348797","journal-title":"ACM Trans. Access. Comput."},{"issue":"1","key":"1194_CR19","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1007\/s10209-014-0399-3","volume":"14","author":"AG Schiavone","year":"2015","unstructured":"Schiavone, A.G., Patern\u00f2, F.: An extensible environment for guideline-based accessibility evaluation of dynamic Web applications. Univ. Access Inf. Soc. 14(1), 111\u2013132 (2015). https:\/\/doi.org\/10.1007\/s10209-014-0399-3","journal-title":"Univ. Access Inf. Soc."},{"key":"1194_CR20","unstructured":"Selenium.: Selenium Web Driver. Retrieved from https:\/\/www.selenium.dev\/"},{"key":"1194_CR21","unstructured":"W3C.: Web Accessibility Evaluation Tools List. https:\/\/www.w3.org\/WAI\/ER\/tools\/ (2024). Accessed 9 Jan, 2024"}],"container-title":["Universal Access in the Information Society"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10209-025-01194-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10209-025-01194-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10209-025-01194-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T11:33:56Z","timestamp":1764934436000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10209-025-01194-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,15]]},"references-count":21,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,8]]}},"alternative-id":["1194"],"URL":"https:\/\/doi.org\/10.1007\/s10209-025-01194-7","relation":{},"ISSN":["1615-5289","1615-5297"],"issn-type":[{"type":"print","value":"1615-5289"},{"type":"electronic","value":"1615-5297"}],"subject":[],"published":{"date-parts":[[2025,2,15]]},"assertion":[{"value":"27 January 2025","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 February 2025","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}