{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T16:25:51Z","timestamp":1783009551833,"version":"3.54.5"},"reference-count":61,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,12,3]],"date-time":"2025-12-03T00:00:00Z","timestamp":1764720000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004054","name":"Deanship of Scientific Research (DSR) at King Abdulaziz University, Jeddah, Saudi Arabia","doi-asserted-by":"publisher","award":["IPP: 1129-612-2025"],"award-info":[{"award-number":["IPP: 1129-612-2025"]}],"id":[{"id":"10.13039\/501100004054","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JTAER"],"abstract":"<jats:p>Effective website usability assessment is crucial for improving user experience, driving customer satisfaction, and ensuring business success, particularly in the competitive e-commerce sector. Traditional methods, such as expert reviews and user testing, are resource-intensive and often fail to fully capture the complex interplay between a site\u2019s aesthetic design and its technical performance. This paper introduces an end-to-end multimodal deep learning framework that automates the usability assessment of fashion e-commerce websites. The framework fuses structured numerical indicators (e.g., load time, mobile compatibility) with high-level visual features extracted from full-page screenshots. The proposed framework employs a comprehensive set of visual backbones\u2014including modern architectures such as ConvNeXt and Vision Transformers (ViT, Swin) alongside established CNNs\u2014and systematically evaluates three fusion strategies: early fusion, late fusion, and a state-of-the-art cross-modal fusion strategy that enables deep, bidirectional interactions between modalities. Extensive experiments demonstrate that the cross-modal fusion approach, particularly when paired with a ConvNeXt backbone, achieves superior performance with a 0.92 accuracy and 0.89 F1-score, outperforming both unimodal and simpler fusion baselines. Model interpretability is provided through SHAP and LIME, confirming that the predictions align with established usability principles and generate actionable insights. Although validated on fashion e-commerce sites, the framework is highly adaptable to other domains\u2014such as e-learning and e-government\u2014via domain-specific data and light fine-tuning. It provides a robust, explainable benchmark for data-driven, multimodal website usability assessment and paves the way for more intelligent, automated user-experience optimization.<\/jats:p>","DOI":"10.3390\/jtaer20040343","type":"journal-article","created":{"date-parts":[[2025,12,3]],"date-time":"2025-12-03T08:44:28Z","timestamp":1764751468000},"page":"343","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Multimodal Deep Learning Framework for Automated Usability Evaluation of Fashion E-Commerce Sites"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4683-3716","authenticated-orcid":false,"given":"Nahed","family":"Alowidi","sequence":"first","affiliation":[{"name":"Department of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,12,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"012006","DOI":"10.1088\/1757-899X\/803\/1\/012006","article-title":"Web-based knowledge management system for Camarines Norte State College","volume":"803","author":"Maligat","year":"2020","journal-title":"IOP Conf. Ser. Mater. Sci. Eng."},{"key":"ref_2","unstructured":"Khalil, N.S., Do\u011fruer, E., Elosta, A.K.O., Eraslan, S., Ye\u015filada, Y., and Harper, S. (2020, January 11\u201314). EyeCrowdata: Towards a web-based crowdsourcing platform for web-related eye-tracking data. Proceedings of the ACM Symposium on Eye Tracking Research and Applications (ETRA \u201920 Adjunct), Stockholm, Sweden."},{"key":"ref_3","unstructured":"Nielsen, J., and Loranger, H. (2006). Prioritizing Web Usability, New Riders."},{"key":"ref_4","first-page":"117","article-title":"A usability analysis of company websites","volume":"48","author":"Cappel","year":"2007","journal-title":"J. Comput. Inf. Syst."},{"key":"ref_5","unstructured":"(2018). Ergonomics of Human\u2013System Interaction\u2014Part 11: Usability: Definitions and Concepts (Standard No. ISO 9241-11:2018)."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sivaji, A., and Tsuaan, S.S. (2012, January 3\u20135). Website user experience (UX) testing tool development using open-source software (OSS). Proceedings of the 2012 Southeast Asian Network of Ergonomics Societies (SEANES), Kuala Lumpur, Malaysia.","DOI":"10.1109\/SEANES.2012.6299576"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lowry, P.B., Spaulding, T., Wells, T., Moody, G., Moffitt, K., and Madariaga, S. (2006, January 4\u20137). A theoretical model and empirical results linking website interactivity and usability satisfaction. Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS \u201906), Kauai, HI, USA.","DOI":"10.1109\/HICSS.2006.33"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"479286","DOI":"10.1155\/2014\/479286","article-title":"An intelligent framework for website usability","volume":"2014","author":"Dingli","year":"2014","journal-title":"Adv. Hum. Comput. Interact"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1109\/52.903167","article-title":"A global perspective on website usability","volume":"18","author":"Becker","year":"2001","journal-title":"IEEE Softw."},{"key":"ref_10","first-page":"292","article-title":"A review of automated website usability evaluation tools: Research issues and challenges","volume":"Volume 12779","author":"Soares","year":"2021","journal-title":"Lecture Notes in Computer Science, Proceedings of the Design, User Experience, and Usability: UX Research and Design (HCII 2021), Virtual Event, 24\u201329 July 2021"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Marenkov, J., Robal, T., and Kalja, A. (2018, January 25\u201327). Guideliner: A tool to improve web UI development for better usability. Proceedings of the 8th International Conference on Web Intelligence, Mining and Semantics, Novi Sad, Serbia.","DOI":"10.1145\/3227609.3227667"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"561","DOI":"10.13052\/jwe1540-9589.17676","article-title":"Auto-extraction and integration of metrics for web user interfaces","volume":"17","author":"Bakaev","year":"2018","journal-title":"J. Web Eng."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.ijhcs.2018.11.006","article-title":"Webthetics: Quantifying webpage aesthetics with deep learning","volume":"124","author":"Dou","year":"2019","journal-title":"Int. J. Hum.-Comput. Stud."},{"key":"ref_14","unstructured":"Hakami, A., Alqarni, R., Muqaibil, A., and Alowidi, N. (2024). Intelligent usability evaluation for fashion websites. arXiv."},{"key":"ref_15","first-page":"169","article-title":"Improving the performance assessment of government web portals: A proposal using data envelopment analysis (DEA)","volume":"18","author":"Luna","year":"2013","journal-title":"Inf. Polity"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"127","DOI":"10.5267\/j.ijdns.2020.2.004","article-title":"Usability evaluation of e-government websites: A case study from Taiwan","volume":"4","author":"Chang","year":"2020","journal-title":"Int. J. Data Netw. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"949","DOI":"10.1007\/s10209-019-00704-8","article-title":"Accessibility and usability analysis of Indian e-government websites","volume":"19","author":"Paul","year":"2020","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1007\/s10209-020-00716-9","article-title":"Accessibility, usability, and security evaluation of Hungarian government websites","volume":"20","author":"Csontos","year":"2021","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_19","first-page":"160975","article-title":"A genetic algorithm-based support vector machine approach for intelligent usability assessment of m-learning applications","volume":"2022","author":"Asghar","year":"2022","journal-title":"Mob. Inf. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Singh, K.N., Samui, A., Mukul, M., Misra, C., and Goswami, B. (2024, January 27\u201329). Usability evaluation of e-learning platforms using UX\/UI design and ML techniques. Proceedings of the 2024 International Conference on Advancements in Smart, Secure and Intelligent Computing (ASSIC), Bhubaneswar, India.","DOI":"10.1109\/ASSIC60049.2024.10507912"},{"key":"ref_21","first-page":"40","article-title":"Integrated usability evaluation framework for university websites","volume":"8","author":"Adepoju","year":"2019","journal-title":"i-Manag. J. Inf. Technol."},{"key":"ref_22","unstructured":"Adepoju, S.A. (2021). Integrated Website Usability Evaluation Model Using Fuzzy Analytical Hierarchy Process and Artificial Neural Network. [Ph.D. Thesis, Federal University of Technology Minna Institutional]."},{"key":"ref_23","first-page":"43","article-title":"The usability of university websites: A study on Sri Lankan universities","volume":"1","author":"Jayakody","year":"2022","journal-title":"Sri Lankan J. Appl. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"49","DOI":"10.24042\/ijecs.v2i2.15045","article-title":"Usability methodologies and data selection: Assessing the usability techniques on educational websites","volume":"2","author":"Abdulrahman","year":"2022","journal-title":"Int. J. Electron. Commun. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1883","DOI":"10.1007\/s10209-024-01160-9","article-title":"Evaluation of the accessibility and usability of university websites: A comparative study of the Gulf region","volume":"24","author":"Fakrudeen","year":"2025","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1007\/s10209-022-00869-9","article-title":"Accessibility, usability, and security evaluation of universities\u2019 prospective student web pages: A comparative study of Europe, North America, and Oceania","volume":"22","author":"Peker","year":"2023","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4018\/IJITWE.333638","article-title":"Evaluating the user interface and usability approaches for e-learning systems","volume":"18","author":"Alqurni","year":"2023","journal-title":"Int. J. Inf. Technol. Web Eng."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Kumar, B., Roy, S., Sinha, A., Iwendi, C., and Str\u00e1\u017eovsk\u00e1, \u013d. (2022). E-commerce website usability analysis using association rule mining and machine learning algorithms. Mathematics, 11.","DOI":"10.3390\/math11010025"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Mailangkay, A., Satyabodhi, R., and Halim, E. (2023, January 15\u201317). Analyzing factors influencing adoption of AI in e-commerce UI\/UX design using TAM. Proceedings of the 2023 IEEE 3rd International Conference on Social Sciences and Intelligence Management (SSIM), Taichung, Taiwan.","DOI":"10.1109\/SSIM59263.2023.10468833"},{"key":"ref_30","first-page":"1237","article-title":"Website usability and user experience during shopping online from abroad","volume":"13","year":"2018","journal-title":"J. Appl. Econ. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Arshad, U., Yasir, T., and Shakeel, T. (2021, January 9\u201310). Web usability and user experience for Pakistani e-commerce websites. Proceedings of the 2021 International Conference on Innovative Computing (ICIC), Lahore, Pakistan.","DOI":"10.1109\/ICIC53490.2021.9693036"},{"key":"ref_32","unstructured":"\u015eim\u015fekler, S. (2024). The Effects of Service Design on Super-App Brand Perception and User Experience. [Master\u2019s Thesis, Middle East Technical University]."},{"key":"ref_33","unstructured":"Shaba, C.D. (2023). Development of e-Banking Website Quality Evaluation Model Using Fuzzy Analytic Hierarchy Process. [Master\u2019s Thesis, Federal University of Technology Minna Institutional]."},{"key":"ref_34","first-page":"325","article-title":"Predicting internet banking effectiveness using artificial neural networks","volume":"12","year":"2021","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1077","DOI":"10.18576\/isl\/110408","article-title":"Evaluation of the contents of the five-star hotel website and customer orientation","volume":"11","author":"Jawabreh","year":"2022","journal-title":"Inf. Sci. Lett."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Biyyapu, V.P., Jammalamadaka, S.K.R., Jammalamadaka, S.B., Chokara, B., Duvvuri, B.K.K., and Budaraju, R.R. (2023). Building an expert system through machine learning for predicting website quality based on its completion. Computers, 12.","DOI":"10.20944\/preprints202308.0161.v1"},{"key":"ref_37","first-page":"36","article-title":"Performance evaluation of websites using machine learning","volume":"51","author":"Ghattas","year":"2020","journal-title":"EIMJ"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Ghattas, M.M., Mora, A.M., and Odeh, S. (2025). A novel approach for evaluating web page performance based on machine learning and optimization algorithms. AI, 6.","DOI":"10.3390\/ai6020019"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Massaro, A., Giannone, D., Birardi, V., and Galiano, A.M. (2021). An innovative approach for the evaluation of web page impact combining user experience and neural network score. Future Internet, 13.","DOI":"10.3390\/fi13060145"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1016\/j.procs.2022.08.036","article-title":"Automated evaluation tools for web and mobile accessibility: Proposal of a new adaptive interface tool","volume":"204","author":"Dias","year":"2022","journal-title":"Procedia Comput. Sci."},{"key":"ref_41","unstructured":"Lu, Y., Yao, B., Gu, H., Huang, J., Wang, J., Li, Y., and Wang, D. (2025). UXAgent: A system for simulating usability testing of web design with LLM agents. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"9752","DOI":"10.1080\/10447318.2024.2427978","article-title":"Unmoderated usability studies evolved: Can GPT ask useful follow-up questions?","volume":"41","author":"Kuric","year":"2024","journal-title":"Int. J. Hum. Comput. Interact."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1016\/j.future.2020.12.014","article-title":"Aesthetic assessment of website design based on multimodal fusion","volume":"117","author":"Liu","year":"2021","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"103019","DOI":"10.1016\/j.ijhcs.2023.103019","article-title":"Calista: A deep learning-based system for understanding and evaluating website aesthetics","volume":"175","author":"Delitzas","year":"2023","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Sun, Y., Liu, G., Han, X., Zuo, W., and Liu, W. (2023, January 17\u201321). FusionNet: An effective network phishing website detection framework based on multi-modal fusion. Proceedings of the 2023 IEEE International Conference on High Performance Computing & Communications, Data Science & Systems, Smart City & Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC\/DSS\/SmartCity\/DependSys), Melbourne, Australia.","DOI":"10.1109\/HPCC-DSS-SmartCity-DependSys60770.2023.00071"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"120537","DOI":"10.1016\/j.eswa.2023.120537","article-title":"Multi-modal fusion using fine-tuned self-attention and transfer learning for veracity analysis of web information","volume":"229","author":"Meel","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_47","unstructured":"Lee, K., Joshi, M., Turc, I.R., Hu, H., Liu, F., Eisenschlos, J.M., Khandelwal, U., Shaw, P., Chang, M.W., and Toutanova, K. (2023, January 3\u20137). Pix2Struct: Screenshot parsing as pretraining for visual language understanding. Proceedings of the International Conference on Machine Learning (ICML), Honolulu, HI, USA."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Xu, H., Chen, L., Zhao, Z., Ma, D., Cao, R., Zhu, Z., and Yu, K. (2024, January 4). Hierarchical multimodal pre-training for visually rich webpage understanding. Proceedings of the 17th ACM International Conference on Web Search and Data Mining (WSDM), Merida, Mexico.","DOI":"10.1145\/3616855.3635753"},{"key":"ref_49","unstructured":"Baechler, G., Sunkara, S., Wang, M., Zubach, F., Mansoor, H., Etter, V., C\u0103rbune, V., Lin, J., Chen, J., and Sharma, A. (2024, January 3\u20139). ScreenAI: A vision-language model for UI and infographics understanding. Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (IJCAI-24), Jeju, Republic of Korea."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Koh, J.Y., Lo, R., Jang, L., Duvvur, V., Lim, M.C., Huang, P.Y., Neubig, G., Zhou, S., Salakhutdinov, R., and Fried, D. (2024, January 3). VisualWebArena: Evaluating multimodal agents on realistic visual web tasks. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, Bangkok, Thailand.","DOI":"10.18653\/v1\/2024.acl-long.50"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Zhang, T., Wang, Y., Zhu, H., Xin, L., Xiang, C., Zhou, T.H., and Ma, J. (May, January 29). WebQuality: A large-scale multi-modal web page quality assessment dataset with multiple scoring dimensions. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2025), Volume 1: Long Papers, Albuquerque, NM, USA.","DOI":"10.18653\/v1\/2025.naacl-long.25"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Majid, E.S.A., Kamaruddin, N., and Mansor, Z. (2015, January 10\u201311). Adaptation of usability principles in responsive web design technique for e-commerce development. Proceedings of the 2015 International Conference on Electrical Engineering and Informatics (ICEEI), Bali, Indonesia.","DOI":"10.1109\/ICEEI.2015.7352593"},{"key":"ref_53","first-page":"86","article-title":"Usability evaluation methods and principles for the web","volume":"13","author":"Mvungi","year":"2015","journal-title":"Int. J. Comput. Sci. Inf. Secur."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Nielsen, J. (1994, January 24\u201328). Enhancing the explanatory power of usability heuristics. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), Boston, MA, USA.","DOI":"10.1145\/191666.191729"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1016\/S0169-8141(97)00059-0","article-title":"A structured methodology for comparative evaluation of user interface designs using usability criteria and measures","volume":"23","author":"Park","year":"1999","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Singh, T., Malik, S., and Sarkar, D. (2016, January 29\u201330). E-commerce website quality assessment based on usability. Proceedings of the 2016 International Conference on Computing, Communication and Automation (ICCCA), Greater Noida, India.","DOI":"10.1109\/CCAA.2016.7813698"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"1814","DOI":"10.1016\/j.tele.2017.09.004","article-title":"Classification of design parameters for e-commerce websites: A novel fuzzy Kano approach","volume":"34","author":"Ilbahar","year":"2017","journal-title":"Telemat. Inform."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1287\/isre.13.2.168.84","article-title":"Assessing a firm\u2019s web presence: A heuristic evaluation procedure for the measurement of usability","volume":"13","author":"Agarwal","year":"2002","journal-title":"Inf. Syst. Res."},{"key":"ref_59","unstructured":"Similarweb Ltd (2025, November 03). Similarweb. Available online: https:\/\/www.similarweb.com\/."},{"key":"ref_60","unstructured":"Semrush Inc (2025, November 03). Semrush. Available online: https:\/\/www.semrush.com\/."},{"key":"ref_61","unstructured":"Baymard Institute (2025, November 03). Baymard Institute. Available online: https:\/\/baymard.com\/."}],"container-title":["Journal of Theoretical and Applied Electronic Commerce Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/0718-1876\/20\/4\/343\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,3]],"date-time":"2025-12-03T09:24:14Z","timestamp":1764753854000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/0718-1876\/20\/4\/343"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,3]]},"references-count":61,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["jtaer20040343"],"URL":"https:\/\/doi.org\/10.3390\/jtaer20040343","relation":{},"ISSN":["0718-1876"],"issn-type":[{"value":"0718-1876","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,3]]}}}