{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T01:47:08Z","timestamp":1787017628993,"version":"build-2736575974"},"reference-count":77,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2026,5,23]],"date-time":"2026-05-23T00:00:00Z","timestamp":1779494400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"name":"Dutch Ministry of Education, Culture and Science","award":["024.004.022, NWA.1389.20.183, NWA ORC 2020\/21"],"award-info":[{"award-number":["024.004.022, NWA.1389.20.183, NWA ORC 2020\/21"]}]},{"DOI":"10.13039\/501100003246","name":"Dutch Research Council","doi-asserted-by":"crossref","award":["KICH3.LTP.20.006"],"award-info":[{"award-number":["KICH3.LTP.20.006"]}],"id":[{"id":"10.13039\/501100003246","id-type":"DOI","asserted-by":"crossref"}]},{"name":"DPG Media, RTL, Dutch Ministry of Economic Affairs and Climate Policy","award":["LTP KIC 2020\u20132023"],"award-info":[{"award-number":["LTP KIC 2020\u20132023"]}]},{"name":"European Union\u2019s Horizon Europe research and innovation program","award":["101070212"],"award-info":[{"award-number":["101070212"]}]},{"name":"UNITE project funded by the European Union","award":["101201510"],"award-info":[{"award-number":["101201510"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>\n                    In two-sided marketplaces, items compete for user attention, which translates to revenue for suppliers. Item exposure, indicated by the amount of attention items receive in a ranking, can be influenced by factors like position bias. Recent work suggests that inter-item dependencies, such as\n                    <jats:italic toggle=\"yes\">outlier items<\/jats:italic>\n                    in a ranking, also affect item exposure. Outlier items are items that observably deviate from the other items in a ranked list w.r.t. task-specific, presentational features. Understanding outlier items is crucial for determining an item\u2019s exposure distribution.\n                  <\/jats:p>\n                  <jats:p>\n                    In our previous work, we investigated the impact of different presentational features on users\u2019 perception of outlierness in e-commerce search result pages. By modeling the problem as visual search tasks, we compared the observability of three main features: price, star rating, and discount tag. We found that participants perceive these features differently in terms of attention and reaction times. Various factors, such as visual complexity (e.g., shape, color), discriminative item features (e.g., a solitary discount tag), and value range, affect item outlierness. These factors can be categorized into two main classes:\n                    <jats:italic toggle=\"yes\">bottom-up<\/jats:italic>\n                    and\n                    <jats:italic toggle=\"yes\">top-down<\/jats:italic>\n                    . Bottom-up factors are driven by visual properties such as color, contrast, and brightness, while top-down factors are influenced by cognitive processes such as expectations and prior knowledge.\n                  <\/jats:p>\n                  <jats:p>In this extension of our previous work, we deepen our analysis of user perceptions of outliers. In particular, we focus on two key questions left unanswered by our previous work: (i) What is the effect of isolated bottom-up visual factors on item outlierness in product lists? (ii) How do top-down factors influence users\u2019 perception of item outlierness in a realistic online shopping scenario?<\/jats:p>\n                  <jats:p>We start with bottom-up factors and employ visual saliency models to evaluate their ability to detect outlier items in product lists purely based on visual attributes. Then, to examine top-down factors, we conduct eye-tracking experiments on the same task as our previous visual search experiment: online shopping. This time, we design the task as a simulated e-commerce environment, mimicking a popular European online shopping platform to be more representative of real-world scenarios. Moreover, we employ eye-tracking to not only be closer to the real-world case but also to address the accuracy problem of reaction time in the visual search task. In our experiments, participants interact with realistic product lists, some containing outliers w.r.t. different presentational features, such as image, price, and discount tag, at different positions.<\/jats:p>\n                  <jats:p>Our experiments show the ability of visual saliency models to detect bottom-up factors, consistently highlighting areas with strong visual contrasts and attention hotspots. While the well-known Itti and Koch model detects general visual attention patterns in an image, a graph-based visual saliency (GBVS) model identifies visual anomalies more effectively. However, one should be cautious about the limitations of these models. Visual saliency models only rely on bottom-up factors, making them naive in that they do not distinguish between separate product features or compare them against each other.<\/jats:p>\n                  <jats:p>The results of our eye-tracking experiment for lists without outliers show that despite being less visually attractive, product descriptions captured attention the fastest, indicating the importance of top-down factors and user knowledge of the task. Our observations in lists with visual outliers suggest that outliers and their immediate neighbors attracted attention faster (in terms of time to first fixation), which is in line with our findings from the visual search task. However, in our eye-tracking experiments, we observed that outlier items engaged users for longer durations (in terms of fixation count and time spent) compared to non-outlier items. This effect was consistent across different outlier features (image, price, discount tag) and various positions within the list.<\/jats:p>","DOI":"10.1145\/3806228","type":"journal-article","created":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T13:07:22Z","timestamp":1776517642000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Understanding Visual Saliency of Outlier Items in Product Search"],"prefix":"10.1145","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6284-1636","authenticated-orcid":false,"given":"Fatemeh","family":"Sarvi","sequence":"first","affiliation":[{"name":"Albert Heijn B.V., Zaandam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9447-4172","authenticated-orcid":false,"given":"Mohammad","family":"Aliannejadi","sequence":"additional","affiliation":[{"name":"University of Amsterdam, Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4722-5840","authenticated-orcid":false,"given":"Sebastian","family":"Schelter","sequence":"additional","affiliation":[{"name":"BIFOLD and TU Berlin, Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1086-0202","authenticated-orcid":false,"given":"Maarten de","family":"Rijke","sequence":"additional","affiliation":[{"name":"University of Amsterdam, Amsterdam, Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,5,23]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206596"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313697"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1108\/JCM-10-2015-1587"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-24085-0_70"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1038\/nrn2795"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3406522.3446023"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210063"},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","unstructured":"Stephanie Brandl Oliver Eberle Tiago Ribeiro Anders S\u00f8gaard and Nora Hollenstein. 2024. Evaluating webcam-based gaze data as an alternative for human rationale annotations. arXiv:2402.19133. Retrieved from https:\/\/arxiv.org\/abs\/2402.19133","DOI":"10.63317\/2u3b4pei3y5h"},{"key":"e_1_3_2_10_2","volume-title":"How People Look at Pictures: A Study of the Psychology and Perception in Art","author":"Buswell Guy Thomas","year":"1935","unstructured":"Guy Thomas Buswell. 1935. How People Look at Pictures: A Study of the Psychology and Perception in Art. Univ. Chicago Press."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1207\/S15327663JCP13-1&2_16"},{"key":"e_1_3_2_12_2","first-page":"257","article-title":"Attention to product images in an online retailing store: An eye-tracking study considering consumer goals and type of product","volume":"23","author":"Chocarro Eguaras Raquel","year":"2022","unstructured":"Raquel Chocarro Eguaras, M\u00f3nica Corti\u00f1as Ugalde, and Arantxa Villanueva Larre. 2022. Attention to product images in an online retailing store: An eye-tracking study considering consumer goals and type of product. Journal of Electronic Commerce Research 23, 4 (2022), 257\u2013281.","journal-title":"Journal of Electronic Commerce Research"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3411962"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.96.3.433"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1080\/00401706.1999.10485670"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.3390\/foods_2020-08124"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.bandc.2019.103582"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","unstructured":"Susann Fiedler Michael Schulte-Mecklenbeck Frank Renkewitz and Jacob L. Orquin. 2020. Guideline for reporting standards of eye-tracking research in decision sciences. PsyArXiv:f6qcy. Retrieved from 10.31234\/osf.io\/f6qcy","DOI":"10.31234\/osf.io\/f6qcy"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1167\/8.2.6"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.visinf.2022.03.001"},{"key":"e_1_3_2_21_2","doi-asserted-by":"crossref","unstructured":"Malay Haldar Hongwei Zhang Kedar Bellare Sherry Chen Soumyadip Banerjee Xiaotang Wang Mustafa Abdool Huiji Gao Pavan Tapadia Liwei He et al. 2024. Learning to rank for maps at Airbnb. arXiv:2407.00091. Retrieved from https:\/\/arxiv.org\/abs\/2407.00091","DOI":"10.1145\/3637528.3671648"},{"key":"e_1_3_2_22_2","volume-title":"Advances in Neural Information Processing Systems","author":"Harel Jonathan","year":"2006","unstructured":"Jonathan Harel, Christof Koch, and Pietro Perona. 2006. Graph-based visual saliency. In Advances in Neural Information Processing Systems, Vol. 19."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.im.2014.05.002"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.4249\/scholarpedia.3327"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/34.730558"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/11731139_68"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076063"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3018661.3018699"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1108\/INTR-11-2018-0477"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1002\/mar.20204"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357127"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.2019.3546"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379503.3403557"},{"issue":"3","key":"e_1_3_2_34_2","first-page":"85","article-title":"Product category dependent consumer preferences for online and offline shopping features and their influence on multi-channel retail alliances","volume":"4","author":"Levin Aron M.","year":"2003","unstructured":"Aron M. Levin, Irwin R. Levin, and C. Edward Heath. 2003. Product category dependent consumer preferences for online and offline shopping features and their influence on multi-channel retail alliances. Journal of Electronic Commerce Research 4, 3 (2003), 85\u201393.","journal-title":"Journal of Electronic Commerce Research"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1080\/0144929X.2020.1761450"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM50108.2020.00135"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33786-4_24"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2016.08.037"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.psycom.2022.100100"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1037\/0096-1523.25.6.1517"},{"key":"e_1_3_2_41_2","first-page":"1","article-title":"Kruskal-wallis test","author":"McKight Patrick E.","year":"2010","unstructured":"Patrick E. McKight and Julius Najab.2010. Kruskal-wallis test. Journal of the Corsini Encyclopedia of Psychology (2010), 1\u201310.","journal-title":"Journal of the Corsini Encyclopedia of Psychology"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3272027"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401100"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00426-002-0109-2"},{"key":"e_1_3_2_45_2","volume-title":"Workshop on Adaptable Research Methods for Empirical Research with Map Users, Virtual Workshop","volume":"6","author":"Murali Madhumitha","year":"2021","unstructured":"Madhumitha Murali and Arzu \u00c7\u00f6ltekin. 2021. Conducting eye tracking studies online. In Workshop on Adaptable Research Methods for Empirical Research with Map Users, Virtual Workshop, Vol. 6."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1086\/259630"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3020165.3020185"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380255"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1037\/a0020747"},{"key":"e_1_3_2_50_2","volume-title":"25th International Joint Conference on Artificial Intelligence (IJCAI \u201916)","author":"Papoutsaki Alexandra","year":"2016","unstructured":"Alexandra Papoutsaki, Patsorn Sangkloy, James Laskey, Nediyana Daskalova, Jeff Huang, and James Hays. 2016. WebGazer: Scalable webcam eye tracking using user interactions. In 25th International Joint Conference on Artificial Intelligence (IJCAI \u201916)."},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/342009.335437"},{"key":"e_1_3_2_52_2","first-page":"553","volume-title":"World Wide Web Conference","author":"Sapiezynski Piotr","year":"2019","unstructured":"Piotr Sapiezynski, Wesley Zeng, Ronald E. Robertson, Alan Mislove, and Christo Wilson. 2019. Quantifying the impact of user attention on fair group representation in ranked lists. In World Wide Web Conference, 553\u2013562."},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3488560.3498441"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3591745"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1162\/089976601750264965"},{"issue":"2","key":"e_1_3_2_56_2","first-page":"116","article-title":"Understanding customer experience","volume":"85","author":"Schwager Andre","year":"2007","unstructured":"Andre Schwager and Christopher Meyer. 2007. Understanding customer experience. Harvard Business Review 85, 2 (2007), 116.","journal-title":"Harvard Business Review"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2009.5204207"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0087415"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.1120.1625"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3220088"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3454774"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1167\/7.14.4"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1080\/02724988843000104"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0285(80)90005-5"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijinfomgt.2020.102294"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3412031"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2016.0113"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2932769"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2014.02.007"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jretai.2007.03.009"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3517031.3529615"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.3758\/BF03200774"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1111\/1467-9280.00006"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-vision-091718-015048"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.visres.2009.11.002"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1145\/3437963.3441751"},{"key":"e_1_3_2_77_2","unstructured":"Himank Yadav Zhengxiao Du and Thorsten Joachims. 2019. Policy-gradient training of fair and unbiased ranking functions. arXiv:1911.08054. Retrieved from https:\/\/arxiv.org\/abs\/1911.08054"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772793"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3806228","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,23]],"date-time":"2026-05-23T12:29:40Z","timestamp":1779539380000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3806228"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,23]]},"references-count":77,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3806228"],"URL":"https:\/\/doi.org\/10.1145\/3806228","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,23]]},"assertion":[{"value":"2025-05-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-06","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}