{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T05:46:50Z","timestamp":1792820810531,"version":"4.5.0"},"reference-count":43,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,1,12]],"date-time":"2020-01-12T00:00:00Z","timestamp":1578787200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>With the growth of e-services in the past two decades, the concept of web accessibility has been given attention to ensure that every individual can benefit from these services without any barriers. Web accessibility is considered one of the main factors that should be taken into consideration while developing webpages. Web Content Accessibility Guidelines 2.0 (WCAG 2.0) have been developed to guide web developers to ensure that web contents are accessible for all users, especially disabled users. Many automatic tools have been developed to check the compliance of websites with accessibility guidelines such as WCAG 2.0 and to help web developers and content creators with designing webpages without barriers for disabled people. Despite the popularity of accessibility evaluation tools in practice, there is no systematic way to compare the performance of web accessibility evaluators. This paper first presents two novel frameworks. The first one is proposed to compare the performance of web accessibility evaluation tools in detecting web accessibility issues based on WCAG 2.0. The second framework is utilized to evaluate webpages in meeting these guidelines. Six homepages of Saudi universities were chosen as case studies to substantiate the concept of the proposed frameworks. Furthermore, two popular web accessibility evaluators, Wave and SiteImprove, are selected to compare their performance. The outcomes of studies conducted using the first proposed framework showed that SiteImprove outperformed WAVE. According to the outcomes of the studies conducted, we can conclude that web administrators would benefit from the first framework in selecting an appropriate tool based on its performance to evaluate their websites based on accessibility criteria and guidelines. Moreover, the findings of the studies conducted using the second proposed framework showed that the homepage of Taibah University is more accessible than the homepages of other Saudi universities. Based on the findings of this study, the second framework can be used by web administrators and developers to measure the accessibility of their websites. This paper also discusses the most common accessibility issues reported by WAVE and SiteImprove.<\/jats:p>","DOI":"10.3390\/info11010040","type":"journal-article","created":{"date-parts":[[2020,1,13]],"date-time":"2020-01-13T04:05:51Z","timestamp":1578888351000},"page":"40","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":82,"title":["Comparing Web Accessibility Evaluation Tools and Evaluating the Accessibility of Webpages: Proposed Frameworks"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7974-7638","authenticated-orcid":false,"given":"Abdullah","family":"Alsaeedi","sequence":"first","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Engineering, Taibah University, Medina 42353, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,1,12]]},"reference":[{"key":"ref_1","unstructured":"Caldwell, B., Cooper, M., Reid, L.G.V., and Erheiden, G. (2019, August 20). Web Content Accessibility Guidelines (WCAG) 2.0. Available online: https:\/\/www.w3.org\/TR\/WCAG20\/."},{"key":"ref_2","unstructured":"(2019, September 15). United States Lawsoverview of the Rehabilitation Act of 1973 (Section 504 And 508). Available online: https:\/\/webaim.org\/articles\/laws\/usa\/rehab."},{"key":"ref_3","unstructured":"(2019, September 13). National Government Website Standard, Available online: https:\/\/www.yesser.gov.sa\/EN\/Methodologies\/NATIONAL_ICT_DIGITALSTANDARDS_PRINCIPLES\/Pages\/National-Government-Website-Standard.aspx."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Brewer, J. (2004, January 17\u201322). Web accessibility highlights and trends. Proceedings of the 2004 International Cross-disciplinary Workshop on Web Accessibility (W4A \u201904), New York, NY, USA.","DOI":"10.1145\/990657.990667"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Vigo, M., Brown, J., and Conway, V. (2013, January 13\u201315). Benchmarking web accessibility evaluation tools: measuring the harm of sole reliance on automated tests. Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility, Rio de Janeiro, Brazil.","DOI":"10.1145\/2461121.2461124"},{"key":"ref_6","unstructured":"Lundqvist, S., and Str\u00f6m, J. (2018, September 25). Web Accessibility in E-Learning: Identifying And Solving Accessibility Issues for Wcag 2.0 Conformance in an E-Learning Application. Student Thesis. Available online: http:\/\/urn.kb.se\/resolve?urn=urn:nbn:se:liu:diva-148913."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"439","DOI":"10.4236\/jsea.2016.99029","article-title":"Accessibility evaluation of arabic e-commerce web sites using automated tools","volume":"9","author":"Alshamari","year":"2016","journal-title":"J. Softw. Eng. Appl."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/s40012-014-0040-x","article-title":"Banking websites in india: An accessibility evaluation","volume":"2","author":"Kaur","year":"2014","journal-title":"CSI Trans. ICT"},{"key":"ref_9","first-page":"201","article-title":"Web accessibility evaluation of government websites for people with disabilities in turkey","volume":"4","author":"Vatansever","year":"2016","journal-title":"J. Adv. Manag. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Adepoju, S.A., Shehu, I.S., and Bake, P. (2016). Accessibility evaluation and performance analysis of e-government websites in nigeria. J. Adv. Inf. Technol., 7.","DOI":"10.12720\/jait.7.1.49-53"},{"key":"ref_11","first-page":"978647","article-title":"Accessibility evaluation of dubai e-government websites: Findings and implications","volume":"2013","author":"AlMourad","year":"2013","journal-title":"J. -Gov. Stud. Best Pract."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1007\/s10209-004-0105-y","article-title":"Comparing accessibility evaluation tools: A method for tool effectiveness","volume":"3","author":"Brajnik","year":"2004","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Mart\u00edn, A., Cechich, A., and Rossi, G. (2008). Comparing Approaches to Web Accessibility Assessment. Handbook of Research on Web Information Systems Quality, IGI Global.","DOI":"10.4018\/978-1-59904-847-5.ch011"},{"key":"ref_14","unstructured":"Shadi Abou-Zahra, N.S., and Keen, L. (2018, May 05). Selecting Web Accessibility Evaluation Tools. Available online: https:\/\/www.w3.org\/WAI\/test-evaluate\/tools\/selecting\/."},{"key":"ref_15","unstructured":"Eggert, E., and Abou-Zahra, S. (2018, July 05). Web Accessibility Evaluation Tools List. Available online: https:\/\/www.w3.org\/WAI\/ER\/tools\/index."},{"key":"ref_16","first-page":"73","article-title":"Wcag 2.0 semi-automatic accessibility evaluation system: Design and implementation","volume":"5","year":"2012","journal-title":"Comput. Inf. Sci."},{"key":"ref_17","first-page":"172","article-title":"Web accessibility challenges","volume":"7","author":"Abuaddous","year":"2016","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_18","first-page":"99","article-title":"e-government website accessibility: In-depth evaluation of Saudi Arabia and oman","volume":"3","author":"Abanumy","year":"2005","journal-title":"Electron. J. Gov."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18848\/1832-3669\/CGP\/v07i03\/56204","article-title":"Evaluating web accessibility of university web sites in the kingdom of Saudi Arabia","volume":"7","author":"MasoodRana","year":"2011","journal-title":"Int. J. Technol. Knowl. Soc."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Al-Khalifa, H.S., Al-Kanhal, M., Al-Nafisah, H., Al-soukaih, N., Al-hussain, E., and Al-onzi, M. (2011, January 25\u201327). A pilot study for evaluating arabic websites using automated wcag 2.0 evaluation tools. Proceedings of the 2011 International Conference on Innovations in Information Technology (IIT 2011), Abu Dhabi, United Arab.","DOI":"10.1109\/INNOVATIONS.2011.5893835"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Alahmadi, T., and Drew, S. (2016, January 17\u201320). An evaluation of the accessibility of top-ranking university websites: Accessibility rates from 2005 to 2015. Proceedings of the DEANZ Biennial Conference, Hamilton, New Zealand.","DOI":"10.61468\/jofdl.v21i1.273"},{"key":"ref_22","first-page":"246","article-title":"The expertise effect on web accessibility evaluation methods","volume":"26","author":"Brajnik","year":"2011","journal-title":"Hum.\u2013Comput. Interact."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1108\/01409170810845930","article-title":"Evaluating government website accessibility: Software tool vs. human experts","volume":"31","author":"Hong","year":"2007","journal-title":"Manag. Res. News"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Elkabani, I., Hamandi, L., Zantout, R., and Mansi, S. (2015, January 21\u201323). Toward better web accessibility. Proceedings of the 2015 5th International Conference on Information & Communication Technology and Accessibility (ICTA), Marrakech, Morocco.","DOI":"10.1109\/ICTA.2015.7426915"},{"key":"ref_25","unstructured":"Grantham, J., Grantham, E., and Powers, D. (February, January 31). Website accessibility: An australian view. Proceedings of the Thirteenth Australasian User Interface Conference (AUIC \u201912), Melbourne, Australia. Available online: http:\/\/dl.acm.org\/citation.cfm?id=2512125.2512129."},{"key":"ref_26","unstructured":"Alayed, A.S. (2018). A Framework and Checklist for Localised Web Content Accessibility Guidelines for Arabic University Websites in Saudi Arabia. [Ph.D. Thesis, University of Southampton]."},{"key":"ref_27","unstructured":"Petrie, H., and Kheir, O. (May, January 28). The relationship between accessibility and usability of websites. Proceedings of the SIGCHI Conference on Human Factors In Computing Systems, San Jose, CA, USA."},{"key":"ref_28","first-page":"1","article-title":"The evaluation of accessibility, usability, and user experience","volume":"1","author":"Petrie","year":"2009","journal-title":"Univers. Access Handb."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Petrie, H., Hamilton, F., King, N., and Pavan, P. (2006, January 22\u201327). Remote usability evaluations with disabled people. Proceedings of the SIGCHI Conference on Human Factors In Computing Systems, Montreal, QC, Canada.","DOI":"10.1145\/1124772.1124942"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1007\/s11423-015-9413-6","article-title":"Evaluating e-learning accessibility by automated and student-centered methods","volume":"64","author":"Kumar","year":"2016","journal-title":"Educ. Technol. Res. Dev."},{"key":"ref_31","first-page":"935272","article-title":"Accessibility evaluation on malaysian e-government websites","volume":"2010","author":"Latif","year":"2010","journal-title":"J. E-Gov. Stud. Best Pract."},{"key":"ref_32","unstructured":"Al-Khalifa, H.S. (2008, January 1\u20134). Exploring the accessibility of Saudi Arabia e-government websites: A preliminary results. Proceedings of the 4th International Conference on Theory and Practice of Electronic Governance (ICEGOV \u201910), Cairo, Egypt."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1007\/s10209-010-0215-7","article-title":"The accessibility of Saudi Arabia government web sites: An exploratory study","volume":"11","year":"2012","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1027","DOI":"10.1007\/s10209-016-0495-7","article-title":"Revisiting the accessibility of Saudi Arabia government websites","volume":"16","author":"Baazeem","year":"2017","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_35","first-page":"1127","article-title":"Effectiveness of accessibility and usability of government websites in Saudi Arabia","volume":"4","author":"Khan","year":"2010","journal-title":"Can. J. Pure Appl. Sci."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.20533\/ijds.2040.2570.2015.0134","article-title":"Evaluating the blackboard system based on web accessibility and usability guidelines","volume":"6","author":"Alotaibi","year":"2015","journal-title":"Int. J. Digit. Soc."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Sullivan, T., and Matson, R. (2000, January 16\u201317). Barriers to use: Usability and content accessibility on the web\u2019s most popular sites. Proceedings of the 2000 Conference on Universal Usability (CUU \u201900)y, Arlington, VA, USA.","DOI":"10.1145\/355460.355549"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1394","DOI":"10.1002\/asi.20233","article-title":"Metric for web accessibility evaluation","volume":"56","author":"Parmanto","year":"2005","journal-title":"J. Am. Soc. Inf. Sci. Technol."},{"key":"ref_39","unstructured":"Cluster, W. (2008, January 9\u201311). Unified Web Evaluation Methodology (UWEM 10), 2006. Proceedings of the 11th International Conference (ICCHP 2008), Linz, Austria."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Miesenberger, K., Klaus, J., Zagler, W.L., and Karshmer, A.I. (2006). Interpreting results from large scale automatic evaluation of web accessibility. Computers Helping People with Special Needs, Springer.","DOI":"10.1007\/11788713"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Song, S., Bu, J., Shen, C., Artmeier, A., Yu, Z., and Zhou, Q. (2018, January 23\u201325). Reliability aware web accessibility experience metric. Proceedings of the Internet of Accessible Things (W4A \u201918), Lyon, France.","DOI":"10.1145\/3192714.3192836"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Al-Faries, A., Al-Khalifa, H.S., Al-Razgan, M.S., and Al-Duwais, M. (2013, January 22\u201325). Evaluating the accessibility and usability of top saudi e-government services. Proceedings of the 7th International Conference on Theory and Practice of Electronic Governance (ICEGOV \u201913), Seoul, Korea.","DOI":"10.1145\/2591888.2591898"},{"key":"ref_43","first-page":"33","article-title":"Evaluating the usability and accessibility of lms \u201cblackboard\u201d at king saud university","volume":"9","author":"Alturki","year":"2016","journal-title":"Contemp. Issues Educ. Res."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/1\/40\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:19:32Z","timestamp":1760361572000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/1\/40"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,12]]},"references-count":43,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,1]]}},"alternative-id":["info11010040"],"URL":"https:\/\/doi.org\/10.3390\/info11010040","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,1,12]]}}}