{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T02:21:37Z","timestamp":1782786097320,"version":"3.54.5"},"reference-count":61,"publisher":"Association for Computing Machinery (ACM)","issue":"ETRA","license":[{"start":{"date-parts":[[2024,5,20]],"date-time":"2024-05-20T00:00:00Z","timestamp":1716163200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006374","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["389792660"],"award-info":[{"award-number":["389792660"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2024,5,20]]},"abstract":"<jats:p>Visual highlighting can guide user attention in complex interfaces. However, its effectiveness under limited attentional capacities is underexplored. This paper examines the joint impact of visual highlighting (permanent and dynamic) and dual-task-induced cognitive load on gaze behaviour. Our analysis, using eye-movement data from 27 participants viewing 150 unique webpages reveals that while participants' ability to attend to UI elements decreases with increasing cognitive load, dynamic adaptations (i.e., highlighting) remain attention-grabbing. The presence of these factors significantly alters what people attend to and thus what is salient. Accordingly, we show that state-of-the-art saliency models increase their performance when accounting for different cognitive loads. Our empirical insights, along with our openly available dataset, enhance our understanding of attentional processes in UIs under varying cognitive (and perceptual) loads and open the door for new models that can predict user attention while multitasking.<\/jats:p>","DOI":"10.1145\/3655610","type":"journal-article","created":{"date-parts":[[2024,5,28]],"date-time":"2024-05-28T16:00:05Z","timestamp":1716912005000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Shifting Focus with HCEye: Exploring the Dynamics of Visual Highlighting and Cognitive Load on User Attention and Saliency Prediction"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-8308-9961","authenticated-orcid":false,"given":"Anwesha","family":"Das","sequence":"first","affiliation":[{"name":"Saarland Informatics Campus, Saarland University, Saarbr\u00fccken, Saarland, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5233-2352","authenticated-orcid":false,"given":"Zekun","family":"Wu","sequence":"additional","affiliation":[{"name":"Saarland Informatics Campus, Saarland University, Saarbr\u00fccken, Saarland, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-7044-8957","authenticated-orcid":false,"given":"Iza","family":"Skrjanec","sequence":"additional","affiliation":[{"name":"Language Science and Technology, Saarland University, Saarbr\u00fccken, Saarland, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4168-6099","authenticated-orcid":false,"given":"Anna Maria","family":"Feit","sequence":"additional","affiliation":[{"name":"Saarland Informatics Campus, Saarland University, Saarbr\u00fccken, Saarland, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,5,28]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Tobii AB. [n. d.]. Tobii Pro Lab. https:\/\/www.tobii.com\/"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v067.i01"},{"key":"e_1_2_2_3_1","volume-title":"Neven AM ElSayed, Jillian Dorrian, David P Webb, James A Walsh, Timothy M Simon, Andrew Irlitti, Ross T Smith, Mark Kohler, et al.","author":"Baumeister James","year":"2017","unstructured":"James Baumeister, Seung Youb Ssin, Neven AM ElSayed, Jillian Dorrian, David P Webb, James A Walsh, Timothy M Simon, Andrew Irlitti, Ross T Smith, Mark Kohler, et al. 2017. Cognitive cost of using augmented reality displays. IEEE transactions on visualization and computer graphics 23, 11 (2017), 2378--2388."},{"key":"e_1_2_2_4_1","volume-title":"What do different evaluation metrics tell us about saliency models? IEEE transactions on pattern analysis and machine intelligence 41, 3","author":"Bylinskii Zoya","year":"2018","unstructured":"Zoya Bylinskii, Tilke Judd, Aude Oliva, Antonio Torralba, and Fr\u00e9do Durand. 2018. What do different evaluation metrics tell us about saliency models? IEEE transactions on pattern analysis and machine intelligence 41, 3 (2018), 740--757."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1111\/bjep.12181"},{"key":"e_1_2_2_6_1","volume-title":"Temporal-spatial feature pyramid for video saliency detection. arXiv preprint arXiv:2105.04213","author":"Chang Qinyao","year":"2021","unstructured":"Qinyao Chang and Shiping Zhu. 2021. Temporal-spatial feature pyramid for video saliency detection. arXiv preprint arXiv:2105.04213 (2021)."},{"key":"e_1_2_2_7_1","volume-title":"A High-Quality Landmarked Infrared Eye Video Dataset (IREye4Task): Eye Behaviors, Insights and Benchmarks for Wearable Mental State Analysis","author":"Chen Siyuan","year":"2023","unstructured":"Siyuan Chen and Julien Epps. 2023. A High-Quality Landmarked Infrared Eye Video Dataset (IREye4Task): Eye Behaviors, Insights and Benchmarks for Wearable Mental State Analysis. IEEE Transactions on Affective Computing (2023)."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10648-009-9098-7"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173856"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3526113.3545651"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3379155.3391321"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025599"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v30i4.2268"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1177\/001872088903100206"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_2_16_1","doi-asserted-by":"crossref","unstructured":"Kristian Tangsgaard Hvelplund. 2014. Eye tracking and the translation process: Reflections on the analysis and interpretation of eye-tracking data. (2014).","DOI":"10.6035\/MonTI.2014.ne1.6"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298710"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581096"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2009.5459462"},{"key":"e_1_2_2_20_1","volume-title":"Physiological metrics of mental workload: A review of recent progress. Multiple task performance","author":"Kramer Arthur F","year":"2020","unstructured":"Arthur F Kramer. 2020. Physiological metrics of mental workload: A review of recent progress. Multiple task performance (2020), 279--328."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2017.2710620"},{"key":"e_1_2_2_22_1","first-page":"2007","article-title":"GUIDe saccade detection and smoothing algorithm","volume":"3","author":"Kumar Manu","year":"2007","unstructured":"Manu Kumar. 2007. GUIDe saccade detection and smoothing algorithm. Technical Rep. Stanford CSTR 3 (2007), 2007.","journal-title":"Technical Rep. Stanford CSTR"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1344471.1344488"},{"key":"e_1_2_2_24_1","volume-title":"Distracted and confused?: Selective attention under load. Trends in cognitive sciences 9, 2","author":"Lavie Nilli","year":"2005","unstructured":"Nilli Lavie. 2005. Distracted and confused?: Selective attention under load. Trends in cognitive sciences 9, 2 (2005), 75--82."},{"key":"e_1_2_2_25_1","volume-title":"Hoe Sung Ryu, and Yang Seok Cho","author":"Lee Jieun","year":"2023","unstructured":"Jieun Lee, Nahyun Lee, Jangkyu Ju, Jihwan Chae, Jiyoon Park, Hoe Sung Ryu, and Yang Seok Cho. 2023. Visual Complexity of Head-Up Display in Automobiles Modulates Attentional Tunneling. Human Factors (2023), 00187208231181496."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1518\/001872007X215791"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3379503.3403557"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3332165.3347945"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2817047"},{"key":"e_1_2_2_30_1","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1109\/TPAMI.2010.70","article-title":"Learning to detect a salient object","volume":"33","author":"Liu Tie","year":"2010","unstructured":"Tie Liu, Zejian Yuan, Jian Sun, Jingdong Wang, Nanning Zheng, Xiaoou Tang, and Heung-Yeung Shum. 2010. Learning to detect a salient object. IEEE Transactions on Pattern analysis and machine intelligence 33, 2 (2010), 353--367.","journal-title":"IEEE Transactions on Pattern analysis and machine intelligence"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1177\/14738716211045354"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00248"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501954"},{"key":"e_1_2_2_34_1","volume-title":"Engineering Artificially Intelligent Systems: A Systems Engineering Approach to Realizing Synergistic Capabilities","author":"Nicosia Max","unstructured":"Max Nicosia and Per Ola Kristensson. 2021. Design Principles for AI-Assisted Attention Aware Systems in Human-in-the-Loop Safety Critical Applications. In Engineering Artificially Intelligent Systems: A Systems Engineering Approach to Realizing Synergistic Capabilities. Springer, 230--246."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2307798.2307800"},{"key":"e_1_2_2_36_1","volume-title":"Aalto Interface Metrics (AIM) A Service and Codebase for Computational GUI Evaluation. In Adjunct Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 16--19","author":"Oulasvirta Antti","year":"2018","unstructured":"Antti Oulasvirta, Samuli De Pascale, Janin Koch, Thomas Langerak, Jussi Jokinen, Kashyap Todi, Markku Laine, Manoj Kristhombuge, Yuxi Zhu, Aliaksei Miniukovich, et al. 2018. Aalto Interface Metrics (AIM) A Service and Codebase for Computational GUI Evaluation. In Adjunct Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 16--19."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447319409526098"},{"key":"e_1_2_2_38_1","volume-title":"R: A Language and Environment for Statistical Computing","author":"Team R Core","year":"2022","unstructured":"R Core Team. 2022. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https:\/\/www.R-project.org\/"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1080\/001401399185315"},{"key":"e_1_2_2_40_1","volume-title":"2020 IEEE. In RSJ International Conference on Intelligent Robots and Systems (IROS). 10241--10247","author":"Reddy Navyasri","unstructured":"Navyasri Reddy, Samyak Jain, Pradeep Yarlagadda, and Vineet Gandhi. [n. d.]. Tidying deep saliency prediction architectures. In 2020 IEEE. In RSJ International Conference on Intelligent Robots and Systems (IROS). 10241--10247."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1167\/16.2.7"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1559\/15230406384373"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1559\/15230406384373"},{"key":"e_1_2_2_44_1","volume-title":"Rethinking the role of top-down attention in vision: Effects attributable to a lossy representation in peripheral vision. Frontiers in psychology 3","author":"Rosenholtz Ruth","year":"2012","unstructured":"Ruth Rosenholtz, Jie Huang, and Krista A Ehinger. 2012. Rethinking the role of top-down attention in vision: Effects attributable to a lossy representation in peripheral vision. Frontiers in psychology 3 (2012), 13."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1054972.1055078"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376709"},{"key":"e_1_2_2_47_1","volume-title":"Decision making in aviation","author":"Schriver Angela T","unstructured":"Angela T Schriver, Daniel G Morrow, Christopher D Wickens, and Donald A Talleur. 2017. Expertise differences in attentional strategies related to pilot decision making. In Decision making in aviation. Routledge, 371--386."},{"key":"e_1_2_2_48_1","volume-title":"Proceedings, Part VII 13","author":"Shen Chengyao","year":"2014","unstructured":"Chengyao Shen and Qi Zhao. 2014. Webpage saliency. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6--12, 2014, Proceedings, Part VII 13. Springer, 33--46."},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2022.3209472"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1075\/ts.21013.sil"},{"key":"e_1_2_2_51_1","volume-title":"Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)."},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1167\/11.5.13"},{"key":"e_1_2_2_53_1","volume-title":"Tobias Schreck, and Hanspeter Pfister.","author":"Strobelt Hendrik","year":"2015","unstructured":"Hendrik Strobelt, Daniela Oelke, Bum Chul Kwon, Tobias Schreck, and Hanspeter Pfister. 2015. Guidelines for effective usage of text highlighting techniques. IEEE transactions on visualization and computer graphics 22, 1 (2015), 489--498."},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG"},{"key":"e_1_2_2_55_1","volume-title":"A feature-integration theory of attention. Cognitive psychology 12, 1","author":"Treisman Anne M","year":"1980","unstructured":"Anne M Treisman and Garry Gelade. 1980. A feature-integration theory of attention. Cognitive psychology 12, 1 (1980), 97--136."},{"key":"e_1_2_2_56_1","volume-title":"Proceedings of the Symposium on Eye Tracking Research and Applications","author":"Oleg","year":"2012","unstructured":"Oleg ?pakov. 2012. Comparison of Eye Movement Filters Used in HCI. In Proceedings of the Symposium on Eye Tracking Research and Applications (Santa Barbara, California) (ETRA '12). Association for Computing Machinery, New York, NY, USA, 281--284. https:\/\/doi.org\/10. 1145\/2168556.2168616"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-91845-5"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0277691"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2018.01.076"},{"key":"e_1_2_2_60_1","volume-title":"Attentional tunneling and task management in synthetic vision displays. The international journal of aviation psychology 19, 2","author":"Wickens Christopher D","year":"2009","unstructured":"Christopher D Wickens and Amy L Alexander. 2009. Attentional tunneling and task management in synthetic vision displays. The international journal of aviation psychology 19, 2 (2009), 182--199."},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.4271\/2012-01-0967"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3655610","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3655610","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,23]],"date-time":"2025-08-23T01:23:13Z","timestamp":1755912193000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3655610"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,20]]},"references-count":61,"journal-issue":{"issue":"ETRA","published-print":{"date-parts":[[2024,5,20]]}},"alternative-id":["10.1145\/3655610"],"URL":"https:\/\/doi.org\/10.1145\/3655610","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,20]]},"assertion":[{"value":"2024-05-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}