{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T03:41:41Z","timestamp":1767152501356,"version":"3.48.0"},"reference-count":38,"publisher":"World Scientific Pub Co Pte Ltd","issue":"04","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Semantic Computing"],"published-print":{"date-parts":[[2025,12]]},"abstract":"<jats:p>Modern video-based eye trackers have reached a level of accuracy sufficient for gaze-based human-computer interaction. In many but not all applications, accuracy even allows for replacing a computer mouse with gaze-based pointing. The Multi-Modal Interaction Concept for Efficient Input (M[Formula: see text]ice) tackles accuracy problems with an on-demand hybrid linear-fisheye magnifier. This paper explores how semantic analysis of screen content can be used as part of an alternative solution to the accuracy problem. This solution integrates machine-learning-based near real-time image analysis into the M[Formula: see text]ice interaction concept and adds semantic highlighting to screen elements near the user\u2019s interaction point of gaze. Semantic highlighting identifies clickable screen elements and adds a number as well as a bounding box to each of these elements. This allows users to select elements with a keystroke. It thus combines visual magnification with visual semantic enrichment. The image analysis is based on a machine learning model specifically trained for the respective use case. A user study (n[Formula: see text]=[Formula: see text]20) showed no significant speed difference with the use of semantic highlighting but a 94.3[Formula: see text]% decrease in interaction errors (misclicks) with the use of semantic highlighting compared to the traditional selection and magnification approaches.<\/jats:p>","DOI":"10.1142\/s1793351x25450011","type":"journal-article","created":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T06:23:42Z","timestamp":1759991022000},"page":"569-588","source":"Crossref","is-referenced-by-count":0,"title":["Semantic Highlighting: Using Semantic Context Detection for Visual Guidance in Multimodal Gaze-Keyboard Interaction"],"prefix":"10.1142","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6992-3300","authenticated-orcid":false,"given":"Florian","family":"Eggenkemper","sequence":"first","affiliation":[{"name":"Centre for Cognitive Science & Institute of Psychology, Technical University of Darmstadt, Karolinenplatz 5, Darmstadt, Germany"}]},{"given":"Luca","family":"Feldmann","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, HSW University of Applied Sciences, Am Stockhof 2, Hamelin, Germany"}]},{"given":"Nils","family":"Sontowski","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, HSW University of Applied Sciences, Am Stockhof 2, Hamelin, Germany"}]},{"given":"Anna","family":"Kiel","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, HSW University of Applied Sciences, Am Stockhof 2, Hamelin, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5636-0801","authenticated-orcid":false,"given":"Constantin A.","family":"Rothkopf","sequence":"additional","affiliation":[{"name":"Centre for Cognitive Science & Institute of Psychology, Technical University of Darmstadt, Karolinenplatz 5, Darmstadt, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-2414-4584","authenticated-orcid":false,"given":"Robert","family":"Mertens","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, HSW University of Applied Sciences, Am Stockhof 2, Hamelin, Germany"},{"name":"BHH University of Applied Sciences, Anckelmannstra\u00dfe 10, Hamburg, Germany"}]}],"member":"219","published-online":{"date-parts":[[2025,10,24]]},"reference":[{"key":"S1793351X25450011BIB001","doi-asserted-by":"publisher","DOI":"10.1109\/ISM.2017.72"},{"key":"S1793351X25450011BIB002","doi-asserted-by":"publisher","DOI":"10.1109\/ISM.2017.46"},{"key":"S1793351X25450011BIB003","doi-asserted-by":"publisher","DOI":"10.1109\/ISM46123.2019.00051"},{"key":"S1793351X25450011BIB004","doi-asserted-by":"publisher","DOI":"10.1109\/ISM.2018.00-10"},{"key":"S1793351X25450011BIB005","doi-asserted-by":"publisher","DOI":"10.1109\/ISM55400.2022.00017"},{"key":"S1793351X25450011BIB006","doi-asserted-by":"publisher","DOI":"10.1109\/ISM59092.2023.00054"},{"key":"S1793351X25450011BIB007","doi-asserted-by":"publisher","DOI":"10.1109\/ICSC64641.2025.00017"},{"key":"S1793351X25450011BIB008","doi-asserted-by":"publisher","DOI":"10.1145\/354324.354363"},{"key":"S1793351X25450011BIB009","doi-asserted-by":"publisher","DOI":"10.1145\/988672.988684"},{"key":"S1793351X25450011BIB010","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2002.994697"},{"key":"S1793351X25450011BIB011","doi-asserted-by":"publisher","DOI":"10.1145\/775152.775178"},{"key":"S1793351X25450011BIB012","first-page":"36","volume-title":"AAAI-98 Workshop on AI and Information Integration","author":"Fensel D. A.","year":"1998"},{"key":"S1793351X25450011BIB013","unstructured":"P. Ziewer, Flexible and automated production of full-fledged electronic lectures, thesis, Technische Universit\u00e4t M\u00fcnchen (2007), p. 270."},{"key":"S1793351X25450011BIB014","unstructured":"Microsoft, Access keys design guidelines - Windows apps, 2023, https:\/\/learn. microsoft.com\/en-us\/windows\/apps\/design\/input\/access-keys (accessed Apr. 15, 2024)."},{"key":"S1793351X25450011BIB015","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173594"},{"key":"S1793351X25450011BIB016","doi-asserted-by":"publisher","DOI":"10.1145\/985921.986037"},{"key":"S1793351X25450011BIB017","doi-asserted-by":"publisher","DOI":"10.1145\/1743666.1743702"},{"key":"S1793351X25450011BIB018","doi-asserted-by":"publisher","DOI":"10.1145\/1054972.1054994"},{"key":"S1793351X25450011BIB019","doi-asserted-by":"publisher","DOI":"10.1109\/INFVIS.1996.559214"},{"key":"S1793351X25450011BIB020","doi-asserted-by":"publisher","DOI":"10.1145\/503376.503424"},{"key":"S1793351X25450011BIB021","first-page":"207","volume-title":"2005 Digest of Technical Papers. Int. Conf. on Consumer Electronics, 2005. ICCE.","author":"Kim E. Y.","year":"2005"},{"key":"S1793351X25450011BIB022","doi-asserted-by":"publisher","DOI":"10.1109\/IEMBS.2006.260774"},{"key":"S1793351X25450011BIB023","doi-asserted-by":"publisher","DOI":"10.1109\/HSI.2008.4581433"},{"key":"S1793351X25450011BIB024","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240692"},{"key":"S1793351X25450011BIB025","first-page":"203","volume-title":"Proc. of Graphics Interface","author":"Ashmore M.","year":"2005"},{"volume-title":"Encyclopedia with Semantic Computing and Robotic Intelligence","year":"2023","author":"Mertens R.","key":"S1793351X25450011BIB026"},{"key":"S1793351X25450011BIB027","volume-title":"ISO 9241-110:2020 - Ergonomics of Human-System Interaction \u2013 Part 110: Interaction Principles","author":"International Organization for Standardization","year":"2020","edition":"3"},{"key":"S1793351X25450011BIB028","first-page":"121","volume-title":"Human-Computer Interaction - INTERACT\u201903","author":"Hansen J.","year":"2003"},{"key":"S1793351X25450011BIB029","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2022.09.211"},{"key":"S1793351X25450011BIB030","doi-asserted-by":"publisher","DOI":"10.1145\/2168556.2168569"},{"key":"S1793351X25450011BIB031","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.91"},{"key":"S1793351X25450011BIB032","unstructured":"Y. Tian, Q. Ye and D. Doermann, YOLOv12: Attention-centric real-time object detectors, arXiv preprint, arXiv:2502.12524, 2025."},{"key":"S1793351X25450011BIB033","doi-asserted-by":"publisher","DOI":"10.1109\/AIxMM62960.2025.00011"},{"key":"S1793351X25450011BIB034","volume-title":"Design Patterns: Elements of Reusable Object-Oriented Software","author":"Gamma E.","year":"1994","edition":"1"},{"key":"S1793351X25450011BIB035","first-page":"189","volume-title":"Usability Evaluation in Industry","author":"Brooke J.","year":"1996"},{"key":"S1793351X25450011BIB036","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2020.1801173"},{"volume-title":"Quantifying the User Experience: Practical Statistics for User Research","year":"2016","author":"Sauro J.","key":"S1793351X25450011BIB037"},{"key":"S1793351X25450011BIB038","doi-asserted-by":"publisher","DOI":"10.1080\/10447310802205776"}],"container-title":["International Journal of Semantic Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S1793351X25450011","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T03:36:55Z","timestamp":1767152215000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S1793351X25450011"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,24]]},"references-count":38,"journal-issue":{"issue":"04","published-print":{"date-parts":[[2025,12]]}},"alternative-id":["10.1142\/S1793351X25450011"],"URL":"https:\/\/doi.org\/10.1142\/s1793351x25450011","relation":{},"ISSN":["1793-351X","1793-7108"],"issn-type":[{"type":"print","value":"1793-351X"},{"type":"electronic","value":"1793-7108"}],"subject":[],"published":{"date-parts":[[2025,10,24]]}}}