{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,4]],"date-time":"2025-07-04T05:56:02Z","timestamp":1751608562637,"version":"3.41.0"},"reference-count":68,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,6,10]],"date-time":"2023-06-10T00:00:00Z","timestamp":1686355200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput.-Hum. Interact."],"published-print":{"date-parts":[[2023,6,30]]},"abstract":"<jats:p>\n            The aesthetic appeal of a website has strong effects on users\u2019 reactions, appraisals, and even behaviors. However, evaluating website aesthetics through user ratings is resource intensive, and extant models to predict website aesthetics are limited in performance and ability. We contribute a novel and more precise approach to predict website aesthetics that considers rating distributions. Moreover, we use this approach as a baseline model to illustrate how future research might be conducted using predictions instead of participants. Our approach is based on a deep convolutional neural network model and uses innovations in the field of image aesthetic prediction. It was trained with the dataset from Reinecke and Gajos [2014] and was validated using two independent large datasets. The final model reached an unprecedented cross-validated correlation between the ground truth and predicted rating of\n            <jats:italic>LCC<\/jats:italic>\n            = 0.752. We then used the model to successfully replicate prior findings and conduct original research as an illustration for AI-based research.\n          <\/jats:p>","DOI":"10.1145\/3569889","type":"journal-article","created":{"date-parts":[[2022,10,29]],"date-time":"2022-10-29T11:06:55Z","timestamp":1667041615000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Predicting Rating Distributions of Website Aesthetics with Deep Learning for AI-Based Research"],"prefix":"10.1145","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8870-6646","authenticated-orcid":false,"given":"Simon","family":"Eisbach","sequence":"first","affiliation":[{"name":"Organizational and Business Psychology, University of M\u00fcnster"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8156-024X","authenticated-orcid":false,"given":"Fabian","family":"Daugs","sequence":"additional","affiliation":[{"name":"Cologne Institute for Digital Ecosystems, Technische Hochschule K\u00f6ln"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8493-9071","authenticated-orcid":false,"given":"Meinald T.","family":"Thielsch","sequence":"additional","affiliation":[{"name":"Organizational and Business Psychology, University of M\u00fcnster"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8402-4859","authenticated-orcid":false,"given":"Matthias","family":"B\u00f6hmer","sequence":"additional","affiliation":[{"name":"Cologne Institute for Digital Ecosystems, Technische Hochschule K\u00f6ln"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7754-2786","authenticated-orcid":false,"given":"Guido","family":"Hertel","sequence":"additional","affiliation":[{"name":"Organizational and Business Psychology, University of M\u00fcnster"}]}],"member":"320","published-online":{"date-parts":[[2023,6,10]]},"reference":[{"key":"e_1_3_3_2_2","first-page":"35","volume-title":"Human-Computer Interaction","author":"Altaboli Ahamed","year":"2011","unstructured":"Ahamed Altaboli and Yingzi Lin. 2011. Objective and subjective measures of visual aesthetics of website interface design: The two sides of the coin. In Human-Computer Interaction, Julie A. Jacko (Ed.). Springer, 35\u201344."},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1525\/9780520351271"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2019.12.012"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICISCAE48440.2019.221601"},{"key":"e_1_3_3_6_2","author":"Berlyne Daniel","year":"1971","unstructured":"Daniel Berlyne. 1971. Aesthetics and Psychobiology. Appleton-Century-Crofts.","journal-title":"Aesthetics and Psychobiology"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.4159\/harvard.9780674734470"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.7717\/peerj.3440"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.2307\/20650308"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995467"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.17705\/1thci.00046"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2009.12.006"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2014.54"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/0167-9473(87)90048-X"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2018.11.006"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1093\/iwc\/iwv033"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1076\/digc.13.4.193.8672"},{"key":"e_1_3_3_18_2","volume-title":"Vorschule der \u00c4sthetik","author":"Fechner Gustav Theodor","year":"1876","unstructured":"Gustav Theodor Fechner. 1876. Vorschule der \u00c4sthetik. Breitkopf und H\u00e4rtel."},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1001\/archpsyc.1983.01790060110016"},{"issue":"4","key":"e_1_3_3_20_2","first-page":"221","article-title":"Berliner Intelligenzstrukturtest f\u00fcr Jugendliche\u202f: Begabungs- und Hochbegabungsdiagnostik (BIS-HB)","volume":"54","author":"Frenzel Anne","year":"2008","unstructured":"Anne Frenzel and Ulrike Nett. 2008. Berliner Intelligenzstrukturtest f\u00fcr Jugendliche\u202f: Begabungs- und Hochbegabungsdiagnostik (BIS-HB) von A. O. J\u00e4ger et al.\u202f; [Rezension]. Zuerst ersch. in: Diagnostica 54, 4 (January 2008), 221\u2013225.","journal-title":"Zuerst ersch. in: Diagnostica"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2389824"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00960"},{"key":"e_1_3_3_23_2","unstructured":"Le Hou Chen-Ping Yu and Dimitris Samaras. 2017. Squared earth mover's distance-based loss for training deep neural networks. arXiv:1611.05916. Retrieved from https:\/\/arxiv.org\/abs\/1611.05916."},{"key":"e_1_3_3_24_2","unstructured":"Andrew Howard Menglong Zhu Bo Chen D. Kalenichenko W. Wang Tobias Weyand M. Andreetto and H. Adam. 2017. MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861. Retrieved from https:\/\/arxiv.org\/abs\/1704.04861."},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1093\/iwc\/iwy002"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/1095872.1095876"},{"key":"e_1_3_3_27_2"},{"key":"e_1_3_3_28_2","first-page":"675","article-title":"Caffe: Convolutional architecture for fast feature embedding","author":"Jia Yangqing","year":"2014","unstructured":"Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the ACM Conference on Multimedia. 675\u2013678. Retrieved from https:\/\/arxiv.org\/abs\/1408.5093.","journal-title":"Proceedings of the ACM Conference on Multimedia"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2016.7532767"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.303"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICWR.2016.7498445"},{"key":"e_1_3_3_32_2","first-page":"1097","volume-title":"Proceedings of the 25th International Conference on Neural Information Processing Systems","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems. Curran Associates Inc, Red Hook, NY, 1097\u20131105."},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2003.09.002"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1348\/0007126042369811"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2001.937632"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/1959022.1959023"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1080\/01449290500330448"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV45572.2020.9093412"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654927"},{"key":"e_1_3_3_40_2","article-title":"Composition-preserving deep photo aesthetics assessment","author":"Mai Long","year":"2016","unstructured":"Long Mai, Hailin Jin, and Feng Liu. 2016. Composition-preserving deep photo aesthetics assessment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702575"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2010.05.006"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1080\/0144929X.2012.694910"},{"key":"e_1_3_3_44_2","unstructured":"Naila Murray and Albert Gordo. 2017. A deep architecture for unified aesthetic prediction. arXiv:1708.04890. Retrieved from https:\/\/arxiv.org\/abs\/1708.04890."},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-0255(02)00404-8"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995539"},{"key":"e_1_3_3_47_2","unstructured":"PsyWeb. 2020. PsyWeb. Retrieved May 18 2020 from https:\/\/psyweb.uni-muenster.de\/."},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2557052"},{"key":"e_1_3_3_49_2","doi-asserted-by":"crossref","first-page":"2049","DOI":"10.1145\/2470654.2481281","article-title":"Predicting users\u2019 first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness","author":"Reinecke Katharina","year":"2013","unstructured":"Katharina Reinecke, Tom Yeh, Luke Miratrix, Rahmatri Mardiko, Yuechen Zhao, Jenny Liu, and Krzysztof Z. Gajos. 2013. Predicting users\u2019 first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2049\u20132058.","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1606.05386"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2007.02.003"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.intcom.2009.10.004"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2015.02.056"},{"key":"e_1_3_3_54_2","first-page":"1","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan Karen","year":"2015","unstructured":"Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In Proceedings of the 3rd International Conference on Learning Representations. 1\u201314. Retrieved from http:\/\/arxiv.org\/pdf\/1409.1556v6.","journal-title":"Proceedings of the 3rd International Conference on Learning Representations"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2831899"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.1080\/13506280500410949"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1093\/iwc\/iwt033"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.7717\/peerj.6516"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1080\/07370024.2017.1421954"},{"key":"e_1_3_3_61_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2020.1805877"},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340764.3340794"},{"key":"e_1_3_3_63_2","volume-title":"The Encyclopedia of Human-Computer Interaction","author":"Tractinsky Noam","year":"2013","unstructured":"Noam Tractinsky. 2013. Visual aesthetics. In The Encyclopedia of Human-Computer Interaction (2nd ed.). M. Soegaard and R. F. Dam (Eds.). The Interaction Design Foundation.","edition":"2"},{"key":"e_1_3_3_64_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2006.06.009"},{"key":"e_1_3_3_65_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2012.06.003"},{"key":"e_1_3_3_66_2","unstructured":"Webwiki. 2020. Webwiki - Das Bewertungsportal f\u00fcr Websites. Retrieved May 18 2020 from https:\/\/www.webwiki.de\/."},{"key":"e_1_3_3_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126246"},{"key":"e_1_3_3_68_2","doi-asserted-by":"publisher","DOI":"10.1177\/1745691617693393"},{"key":"e_1_3_3_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518703"}],"container-title":["ACM Transactions on Computer-Human Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3569889","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3569889","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:48:58Z","timestamp":1750182538000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3569889"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,10]]},"references-count":68,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,6,30]]}},"alternative-id":["10.1145\/3569889"],"URL":"https:\/\/doi.org\/10.1145\/3569889","relation":{},"ISSN":["1073-0516","1557-7325"],"issn-type":[{"type":"print","value":"1073-0516"},{"type":"electronic","value":"1557-7325"}],"subject":[],"published":{"date-parts":[[2023,6,10]]},"assertion":[{"value":"2021-06-30","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-08-31","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}