{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,27]],"date-time":"2026-06-27T02:44:08Z","timestamp":1782528248379,"version":"3.54.5"},"reference-count":71,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2020,11,8]],"date-time":"2020-11-08T00:00:00Z","timestamp":1604793600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"FONDECYT Regular","award":["1191791"],"award-info":[{"award-number":["1191791"]}]},{"name":"CONICYT, FONDECYT Iniciacion","award":["11150783"],"award-info":[{"award-number":["11150783"]}]},{"name":"ANID - Millennium Science Initiative Program","award":["ICN17_002"],"award-info":[{"award-number":["ICN17_002"]}]},{"name":"Millennium Institute for Foundational Research on Data"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Interact. Intell. Syst."],"published-print":{"date-parts":[[2020,12,31]]},"abstract":"<jats:p>Explaining suggestions made by recommendation systems is key to make users trust and accept these systems. This is specially critical in areas such as art image recommendation. Traditionally, artworks are sold in galleries where people can see them physically, and artists have the chance to persuade the people into buying them. On the other side, online art stores only offer the user the action of navigating through the catalog, but nobody plays the persuading role of the artist. Moreover, few works in recommendation systems provide a perspective of the many variables involved in the user perception of several aspects of the system such as domain knowledge, relevance, explainability, and trust. In this article, we aim to fill this gap by studying several aspects of the user experience with a recommender system of artistic images, from algorithmic and HCI perspectives. We conducted two user studies in Amazon Mechanical Turk to evaluate different levels of explainability, combined with different algorithms. While in study 1 we focus only on a desktop interface, in study 2 we attempt to understand the effect of explanations in mobile devices.<\/jats:p>\n          <jats:p>In general, our experiments confirm that explanations of recommendations in the image domain are useful and increase user satisfaction, perception of explainability and relevance. In the first study, our results show that the observed effects are dependent on the underlying recommendation algorithm used. In the second study, our results show that these effects are also dependent of the device used in the study but with a smaller effect. Finally, using the framework by Knijnenburg et\u00a0al., we provide a comprehensive model, for each study, which synthesizes the effects between different variables involved in the user experience with explainable visual recommender systems of artistic images.<\/jats:p>","DOI":"10.1145\/3369396","type":"journal-article","created":{"date-parts":[[2020,7,7]],"date-time":"2020-07-07T12:34:07Z","timestamp":1594125247000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":17,"title":["Algorithmic and HCI Aspects for Explaining Recommendations of Artistic Images"],"prefix":"10.1145","volume":"10","author":[{"given":"Vicente","family":"Dominguez","sequence":"first","affiliation":[{"name":"Pontificia Universidad Cat\u00f3lica de Chile 8 IMFD, Santiago, Chile"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ivania","family":"Donoso-Guzm\u00e1n","sequence":"additional","affiliation":[{"name":"Pontificia Universidad Cat\u00f3lica de Chile, Santiago, Chile"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pablo","family":"Messina","sequence":"additional","affiliation":[{"name":"Pontificia Universidad Cat\u00f3lica de Chile 8 IMFD, Santiago, Chile"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Denis","family":"Parra","sequence":"additional","affiliation":[{"name":"Pontificia Universidad Cat\u00f3lica de Chile 8 IMFD, Santiago, Chile"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,11,8]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174156"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2481244.2481250"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2018.04.004"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the Annual Conference on Museums and the Web.","author":"Aroyo L. M.","unstructured":"L. M. Aroyo , Y. Wang , R. Brussee , Peter Gorgels , L. W. Rutledge , and N. Stash . 2007. Personalized museum experience: The rijksmuseum use case . In Proceedings of the Annual Conference on Museums and the Web. L. M. Aroyo, Y. Wang, R. Brussee, Peter Gorgels, L. W. Rutledge, and N. Stash. 2007. Personalized museum experience: The rijksmuseum use case. In Proceedings of the Annual Conference on Museums and the Web."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0121595"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/SMC.2015.139"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2365952.2365964"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301275.3302289"},{"key":"e_1_2_1_9_1","volume-title":"Music Recommendation and Discovery","author":"Celma Oscar","unstructured":"Oscar Celma . 2010. Music recommendation . In Music Recommendation and Discovery . Springer , 43--85. Oscar Celma. 2010. Music recommendation. In Music Recommendation and Discovery. Springer, 43--85."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.2190\/EM.28.2.f"},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the 4th International Conference on User Modeling. Citeseer.","author":"Cook Ronny","year":"1994","unstructured":"Ronny Cook and Judy Kay . 1994 . The justified user model: A viewable, explained user model . In Proceedings of the 4th International Conference on User Modeling. Citeseer. Ronny Cook and Judy Kay. 1994. The justified user model: A viewable, explained user model. In Proceedings of the 4th International Conference on User Modeling. Citeseer."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2959100.2959190"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-008-9051-3"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301275.3302274"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3125486.3125495"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080826"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v32i3.2365"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.265"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2013.12.007"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2843948"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484100"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-05579-4_33"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1017\/iop.2015.23"},{"key":"e_1_2_1_25_1","volume-title":"Staveland","author":"Hart Sandra G.","year":"1988","unstructured":"Sandra G. Hart and Lowell E . Staveland . 1988 . Development of NASA-TLX (task load index): Results of empirical and theoretical research. In Advances in Psychology. Vol. 52 . Elsevier , 139--183. Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (task load index): Results of empirical and theoretical research. In Advances in Psychology. Vol. 52. Elsevier, 139--183."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2959100.2959152"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.9973"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/358916.358995"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2856767.2856811"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1080\/10705519909540118"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654889"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357127"},{"key":"e_1_2_1_33_1","unstructured":"G\u00fcnter Klambauer Thomas Unterthiner Andreas Mayr and Sepp Hochreiter. 2017. Self-normalizing neural networks. In Advances in Neural Information Processing Systems. 971--980.  G\u00fcnter Klambauer Thomas Unterthiner Andreas Mayr and Sepp Hochreiter. 2017. Self-normalizing neural networks. In Advances in Neural Information Processing Systems. 971--980."},{"key":"e_1_2_1_34_1","doi-asserted-by":"crossref","unstructured":"B. P. Knijnenburg N. Rao and A. Kobsa. 2012b. Experimental Materials Used in the Study on Inspectability and Control in Social Recommender Systems. Technical Report. Institute for Software Research University of California Irvine.  B. P. Knijnenburg N. Rao and A. Kobsa. 2012b. Experimental Materials Used in the Study on Inspectability and Control in Social Recommender Systems. Technical Report. Institute for Software Research University of California Irvine.","DOI":"10.1145\/2365952.2365966"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2365952.2365966"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-011-9118-4"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-011-9112-x"},{"key":"e_1_2_1_38_1","volume-title":"Hinton","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky , Ilya Sutskever , and Geoffrey E . Hinton . 2012 . Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems . 1097--1105. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 1097--1105."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2013.6645235"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1873951.1873965"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/176789.176792"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-011-0124-6"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766462.2767755"},{"key":"e_1_2_1_44_1","volume-title":"Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work. ACM, 171--180","author":"McNee Sean M.","unstructured":"Sean M. McNee , Nishikant Kapoor , and Joseph A. Konstan . 2006. Don\u2019t look stupid: Avoiding pitfalls when recommending research papers . In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work. ACM, 171--180 . Sean M. McNee, Nishikant Kapoor, and Joseph A. Konstan. 2006. Don\u2019t look stupid: Avoiding pitfalls when recommending research papers. In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work. ACM, 171--180."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-018-9206-9"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2018.07.007"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3297280.3297443"},{"key":"e_1_2_1_48_1","volume-title":"Feature visualization. Distill","author":"Olah Chris","year":"2017","unstructured":"Chris Olah , Alexander Mordvintsev , and Ludwig Schubert . 2017. Feature visualization. Distill ( 2017 ). DOI:http:\/\/dx.doi.org\/10.23915\/distill.00007 10.23915\/distill.00007 Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. 2017. Feature visualization. Distill (2017). DOI:http:\/\/dx.doi.org\/10.23915\/distill.00007"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2015.01.007"},{"key":"e_1_2_1_50_1","volume-title":"Advanced Techniques in Web Intelligence-2","author":"Parra Denis","unstructured":"Denis Parra and Shaghayegh Sahebi . 2013. Recommender systems: Sources of knowledge and evaluation metrics . In Advanced Techniques in Web Intelligence-2 . Springer , 149--175. Denis Parra and Shaghayegh Sahebi. 2013. Recommender systems: Sources of knowledge and evaluation metrics. In Advanced Techniques in Web Intelligence-2. Springer, 149--175."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/2043932.2043962"},{"key":"e_1_2_1_52_1","volume-title":"Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. AUAI Press, 452--461","author":"Rendle Steffen","year":"2009","unstructured":"Steffen Rendle , Christoph Freudenthaler , Zeno Gantner , and Lars Schmidt-Thieme . 2009 . BPR: Bayesian personalized ranking from implicit feedback . In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. AUAI Press, 452--461 . Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. AUAI Press, 452--461."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_2_1_54_1","doi-asserted-by":"crossref","unstructured":"Joel Ross Lilly Irani M. Silberman Andrew Zaldivar and Bill Tomlinson. 2010. Who are the crowdworkers?: Shifting demographics in mechanical turk. In CHI\u201910 Extended Abstracts on Human Factors in Computing Systems. ACM 2863--2872.  Joel Ross Lilly Irani M. Silberman Andrew Zaldivar and Bill Tomlinson. 2010. Who are the crowdworkers?: Shifting demographics in mechanical turk. In CHI\u201910 Extended Abstracts on Human Factors in Computing Systems. ACM 2863--2872.","DOI":"10.1145\/1753846.1753873"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/1526709.1526813"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2362402.2362405"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2014.131"},{"key":"e_1_2_1_58_1","doi-asserted-by":"crossref","unstructured":"Rashmi Sinha and Kirsten Swearingen. 2002. The role of transparency in recommender systems. In CHI\u201902 Extended Abstracts on Human Factors in Computing Systems. ACM 830--831.  Rashmi Sinha and Kirsten Swearingen. 2002. The role of transparency in recommender systems. In CHI\u201902 Extended Abstracts on Human Factors in Computing Systems. ACM 830--831.","DOI":"10.1145\/506443.506619"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/1297231.1297259"},{"key":"e_1_2_1_60_1","volume-title":"Designing and Evaluating Explanations for Recommender Systems","author":"Tintarev Nava","unstructured":"Nava Tintarev and Judith Masthoff . 2011. Designing and Evaluating Explanations for Recommender Systems . Springer , New York, NY , 479--510. http:\/\/dx.doi.org\/10.1007\/978-0-387-85820-3_15 10.1007\/978-0-387-85820-3_15 Nava Tintarev and Judith Masthoff. 2011. Designing and Evaluating Explanations for Recommender Systems. Springer, New York, NY, 479--510. http:\/\/dx.doi.org\/10.1007\/978-0-387-85820-3_15"},{"key":"e_1_2_1_61_1","volume-title":"Recommender Systems Handbook","author":"Tintarev Nava","unstructured":"Nava Tintarev and Judith Masthoff . 2015. Explaining recommendations: Design and evaluation . In Recommender Systems Handbook . Springer , 353--382. Nava Tintarev and Judith Masthoff. 2015. Explaining recommendations: Design and evaluation. In Recommender Systems Handbook. Springer, 353--382."},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1117\/12.650751"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/2449396.2449442"},{"key":"e_1_2_1_64_1","volume-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI","volume":"19","author":"Wang Danding","unstructured":"Danding Wang , Qian Yang , Ashraf Abdul , and Brian Y. Lim . 2019. Designing theory-driven user-centric explainable ai . In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI , Vol. 19 . Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y. Lim. 2019. Designing theory-driven user-centric explainable ai. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI, Vol. 19."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-38844-6_7"},{"key":"e_1_2_1_66_1","volume-title":"But Online Sales Booming. Retrieved","author":"Weinswig Deborah","year":"2016","unstructured":"Deborah Weinswig . 2016 . Art Market Cooling , But Online Sales Booming. Retrieved March 21, 2017 from https:\/\/www.forbes.com\/sites\/deborahweinswig\/2016\/05\/13\/art-market-cooling-but-online-sales-booming\/. Deborah Weinswig. 2016. Art Market Cooling, But Online Sales Booming. Retrieved March 21, 2017 from https:\/\/www.forbes.com\/sites\/deborahweinswig\/2016\/05\/13\/art-market-cooling-but-online-sales-booming\/."},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025171.3025219"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/1386790.1386827"},{"key":"e_1_2_1_69_1","volume-title":"Explainable recommendation: A survey and new perspectives. CoRR abs\/804.11192","author":"Zhang Yongfeng","year":"2018","unstructured":"Yongfeng Zhang and Xu Chen . 2018. Explainable recommendation: A survey and new perspectives. CoRR abs\/804.11192 ( 2018 ). http:\/\/arxiv.org\/abs\/1804.11192 Yongfeng Zhang and Xu Chen. 2018. Explainable recommendation: A survey and new perspectives. CoRR abs\/804.11192 (2018). http:\/\/arxiv.org\/abs\/1804.11192"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/1864708.1864737"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/1060745.1060754"}],"container-title":["ACM Transactions on Interactive Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3369396","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3369396","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:23:30Z","timestamp":1750202610000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3369396"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,8]]},"references-count":71,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,12,31]]}},"alternative-id":["10.1145\/3369396"],"URL":"https:\/\/doi.org\/10.1145\/3369396","relation":{},"ISSN":["2160-6455","2160-6463"],"issn-type":[{"value":"2160-6455","type":"print"},{"value":"2160-6463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,8]]},"assertion":[{"value":"2019-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-11-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}