{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T01:58:22Z","timestamp":1781315902768,"version":"3.54.1"},"reference-count":48,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,3,23]],"date-time":"2021-03-23T00:00:00Z","timestamp":1616457600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","award":["01JD1811B"],"award-info":[{"award-number":["01JD1811B"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","award":["01JD1811C"],"award-info":[{"award-number":["01JD1811C"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Currently an increasing number of head mounted displays (HMD) for virtual and augmented reality (VR\/AR) are equipped with integrated eye trackers. Use cases of these integrated eye trackers include rendering optimization and gaze-based user interaction. In addition, visual attention in VR and AR is interesting for applied research based on eye tracking in cognitive or educational sciences for example. While some research toolkits for VR already exist, only a few target AR scenarios. In this work, we present an open-source eye tracking toolkit for reliable gaze data acquisition in AR based on Unity 3D and the Microsoft HoloLens 2, as well as an R package for seamless data analysis. Furthermore, we evaluate the spatial accuracy and precision of the integrated eye tracker for fixation targets with different distances and angles to the user (n=21). On average, we found that gaze estimates are reported with an angular accuracy of 0.83 degrees and a precision of 0.27 degrees while the user is resting, which is on par with state-of-the-art mobile eye trackers.<\/jats:p>","DOI":"10.3390\/s21062234","type":"journal-article","created":{"date-parts":[[2021,3,23]],"date-time":"2021-03-23T23:59:41Z","timestamp":1616543981000},"page":"2234","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":100,"title":["ARETT: Augmented Reality Eye Tracking Toolkit for Head Mounted Displays"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1052-8901","authenticated-orcid":false,"given":"Sebastian","family":"Kapp","sequence":"first","affiliation":[{"name":"Department of Physics, Technische Universit\u00e4t Kaiserslautern, Erwin-Schr\u00f6dinger-Str. 46, 67663 Kaiserslautern, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6730-2466","authenticated-orcid":false,"given":"Michael","family":"Barz","sequence":"additional","affiliation":[{"name":"German Research Center for Artificial Intelligence (DFKI), Interactive Machine Learning Department, Stuhlsatzenhausweg 3, Saarland Informatics Campus D3_2, 66123 Saarbr\u00fccken, Germany"},{"name":"Applied Artificial Intelligence, Oldenburg University, Marie-Curie Str. 1, 26129 Oldenburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4544-1462","authenticated-orcid":false,"given":"Sergey","family":"Mukhametov","sequence":"additional","affiliation":[{"name":"Department of Physics, Technische Universit\u00e4t Kaiserslautern, Erwin-Schr\u00f6dinger-Str. 46, 67663 Kaiserslautern, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8857-8709","authenticated-orcid":false,"given":"Daniel","family":"Sonntag","sequence":"additional","affiliation":[{"name":"German Research Center for Artificial Intelligence (DFKI), Interactive Machine Learning Department, Stuhlsatzenhausweg 3, Saarland Informatics Campus D3_2, 66123 Saarbr\u00fccken, Germany"},{"name":"Applied Artificial Intelligence, Oldenburg University, Marie-Curie Str. 1, 26129 Oldenburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6985-3218","authenticated-orcid":false,"given":"Jochen","family":"Kuhn","sequence":"additional","affiliation":[{"name":"Department of Physics, Technische Universit\u00e4t Kaiserslautern, Erwin-Schr\u00f6dinger-Str. 46, 67663 Kaiserslautern, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Fairclough, S.H., and Gilleade, K. (2014). Eye Tracking and Eye-Based Human\u2013Computer Interaction. Advances in Physiological Computing, Springer.","DOI":"10.1007\/978-1-4471-6392-3"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Morimoto, C., and Pfeiffer, T. (2018). Advantages of eye-gaze over head-gaze-based selection in virtual and augmented reality under varying field of views. Proceedings of the Workshop on Communication by Gaze Interaction\u2014COGAIN \u201918, ACM Press.","DOI":"10.1145\/3206343"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2366145.2366183","article-title":"Foveated 3D graphics","volume":"31","author":"Guenter","year":"2012","journal-title":"ACM Trans. Graph."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2980179.2980246","article-title":"Towards foveated rendering for gaze-tracked virtual reality","volume":"35","author":"Patney","year":"2016","journal-title":"ACM Trans. Graph."},{"key":"ref_5","unstructured":"Tobii Pro AB (2020, November 12). Pro Lab User Manual. Available online: https:\/\/www.tobiipro.com\/siteassets\/tobii-pro\/user-manuals\/Tobii-Pro-Lab-User-Manual\/?v=1.152."},{"key":"ref_6","unstructured":"Pupil Labs (2020, November 20). Add Awareness to Your VR\/AR Experience: Integrate and React. Available online: https:\/\/pupil-labs.com\/products\/vr-ar\/."},{"key":"ref_7","unstructured":"Tobii VR (2020, November 20). Tobii VR: Discover New Possibilities with Eye Tracking in VR. Available online: https:\/\/vr.tobii.com\/."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Sharif, B., and Krejtz, K. (2018). EyeMR\u2014Low-cost Eye-Tracking for Rapid-prototyping in Head-mounted Mixed Reality. Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications, ACM.","DOI":"10.1145\/3204493"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Lee, K.F., Chen, Y.L., Yu, C.W., Chin, K.Y., and Wu, C.H. (2020). Gaze Tracking and Point Estimation Using Low-Cost Head-Mounted Devices. Sensors, 20.","DOI":"10.3390\/s20071917"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Krejtz, K., and Sharif, B. (2019). EyeMRTK: A Toolkit for Developing Eye Gaze Interactive Applications in Virtual and Augmented Reality. Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, ACM.","DOI":"10.1145\/3314111"},{"key":"ref_11","unstructured":"Bulling, A., Huckauf, A., Jain, E., Radach, R., and Weiskopf, D. (2020). GazeMetrics: An Open-Source Tool for Measuring the Data Quality of HMD-based Eye Trackers. ACM Symposium on Eye Tracking Research and Applications, ACM."},{"key":"ref_12","unstructured":"Magic Leap (2020, November 20). Magic Leap 1: A Thousand Breakthroughs in One. Available online: https:\/\/www.magicleap.com\/en-us\/magic-leap-1."},{"key":"ref_13","unstructured":"Microsoft (2020, November 20). HoloLens 2: A New Reality for Computing. Available online: https:\/\/www.microsoft.com\/en-us\/hololens."},{"key":"ref_14","unstructured":"Microsoft (2020, November 17). Eye Tracking in the Mixed Reality Toolkit. Available online: https:\/\/microsoft.github.io\/MixedRealityToolkit-Unity\/Documentation\/EyeTracking\/EyeTracking_Main.html."},{"key":"ref_15","unstructured":"Magic Leap (2020, November 20). Eye Gaze. Available online: https:\/\/developer.magicleap.com\/en-us\/learn\/guides\/design-eye-gaze."},{"key":"ref_16","unstructured":"Bulling, A., Huckauf, A., Jain, E., Radach, R., and Weiskopf, D. (2020). Positional head-eye tracking outside the lab: An open-source solution. ACM Symposium on Eye Tracking Research and Applications, ACM."},{"key":"ref_17","unstructured":"Holmqvist, K., and Andersson, R. (2011). Eye Tracking: A Comprehensive Guide to Methods, Paradigms and Measures, Lund Eye-Tracking Research Institute."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mardanbegi, D., and Hansen, D.W. (2012). Parallax error in the monocular head-mounted eye trackers. Proceedings of the 2012 ACM Conference on Ubiquitous Computing, ACM.","DOI":"10.1145\/2370216.2370366"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Barz, M., Stauden, S., and Sonntag, D. (2020). Visual Search Target Inference in Natural Interaction Settings with Machine Learning. Proceedings of the 2020 ACM Symposium on Eye Tracking Research & Applications, Association for Computing Machinery.","DOI":"10.1145\/3379155.3391314"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Barz, M., Daiber, F., and Bulling, A. (2016). Prediction of Gaze Estimation Error for Error-Aware Gaze-Based Interfaces. Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications, ACM Press.","DOI":"10.1145\/2857491.2857493"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Holmqvist, K., Nystr\u00f6m, M., and Mulvey, F. (2012). Eye tracker data quality: What it is and how to measure it. Proceedings of the Symposium on Eye Tracking Research and Applications, ACM.","DOI":"10.1145\/2168556.2168563"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Barz, M., Bulling, A., and Daiber, F. (2015). Computational Modelling and Prediction of Gaze Estimation Error for Head-Mounted Eye Trackers, DFKI. Technical Report.","DOI":"10.1145\/2857491.2857493"},{"key":"ref_23","unstructured":"Unity Technologies (2021, February 23). Unity Real-Time Development Platform|3D, 2D VR & AR Engine. Available online: https:\/\/unity.com\/."},{"key":"ref_24","unstructured":"The R Foundation (2021, February 23). R: The R Project for Statistical Computing. Available online: https:\/\/www.r-project.org\/."},{"key":"ref_25","unstructured":"Microsoft (2020, November 17). EyesPose Class. Available online: https:\/\/docs.microsoft.com\/de-de\/uwp\/api\/windows.perception.people.eyespose?view=winrt-19041."},{"key":"ref_26","unstructured":"Microsoft (2020, November 12). Eye Tracking on HoloLens 2. Available online: https:\/\/docs.microsoft.com\/en-us\/windows\/mixed-reality\/design\/eye-tracking."},{"key":"ref_27","unstructured":"Microsoft (2020, November 13). Create Mixed Reality Photos and Videos. Available online: https:\/\/docs.microsoft.com\/en-us\/hololens\/holographic-photos-and-videos."},{"key":"ref_28","unstructured":"Brush, A.J., Friday, A., Kientz, J., Scott, J., and Song, J. (2014). Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-based Interaction. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct Publication\u2014UbiComp \u201914 Adjunct, ACM Press."},{"key":"ref_29","unstructured":"Dink, J., and Ferguson, B. (2020, November 24). eyetrackingR: An R Library for Eye-tracking Data Analysis. Available online: http:\/\/www.eyetracking-r.com\/."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1129","DOI":"10.1177\/0301006615594944","article-title":"ETRAN\u2013R Extension Package for Eye Tracking Results Analysis","volume":"44","author":"Zhegallo","year":"2015","journal-title":"Perception"},{"key":"ref_31","unstructured":"Olsen, A. (2020, November 12). The Tobii I-VT Fixation Filter: Algorithm description. Available online: https:\/\/www.tobiipro.com\/siteassets\/tobii-pro\/learn-and-support\/analyze\/how-do-we-classify-eye-movements\/tobii-pro-i-vt-fixation-filter.pdf\/?v=2012."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Llanes-Jurado, J., Mar\u00edn-Morales, J., Guixeres, J., and Alca\u00f1iz, M. (2020). Development and Calibration of an Eye-Tracking Fixation Identification Algorithm for Immersive Virtual Reality. Sensors, 20.","DOI":"10.3390\/s20174956"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Salvucci, D.D., and Goldberg, J.H. (2000). Identifying Fixations and Saccades in Eye-Tracking Protocols. Proceedings of the Eye Tracking Research & Applications Symposium 2000 Palm Beach Gardens, FL, November 6\u20138, 2000, Association for Computing Machinery.","DOI":"10.1145\/355017.355028"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Mark, G., Fussell, S., Lampe, C., Schraefel, M., Hourcade, J.P., Appert, C., and Wigdor, D. (2017). Toward Everyday Gaze Input. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, ACM.","DOI":"10.1145\/3025453"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Steil, J., Huang, M.X., and Bulling, A. (2018). Fixation detection for head-mounted eye tracking based on visual similarity of gaze targets. Eye Tracking Research and Applications Symposium (ETRA), Association for Computing Machinery.","DOI":"10.1145\/3204493.3204538"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"573","DOI":"10.3758\/BF03195486","article-title":"3-D eye movement analysis","volume":"34","author":"Duchowski","year":"2002","journal-title":"Behav. Res. Methods Instrum. Comput."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1119\/1.4999739","article-title":"Augmenting the thermal flux experiment: A mixed reality approach with the HoloLens","volume":"55","author":"Strzys","year":"2017","journal-title":"Phys. Teach."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1119\/1.5084931","article-title":"Augmenting Kirchhoff\u2019s laws: Using augmented reality and smartglasses to enhance conceptual electrical experiments for high school students","volume":"57","author":"Kapp","year":"2019","journal-title":"Phys. Teach."},{"key":"ref_39","first-page":"282","article-title":"Using Eye-Gaze and Visualization to Augment Memory","volume":"Volume 8530 LNCS","author":"Streitz","year":"2014","journal-title":"Distributed, Ambient, and Pervasive Interactions"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Toyama, T., Sonntag, D., Orlosky, J., and Kiyokawa, K. (2015). Attention Engagement and Cognitive State Analysis for Augmented Reality Text Display Functions. Proceedings of the 20th International Conference on Intelligent User Interfaces\u2014IUI \u201915, ACM Press.","DOI":"10.1145\/2678025.2701384"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Toyama, T., Orlosky, J., Sonntag, D., and Kiyokawa, K. (2014). A Natural Interface for Multi-Focal Plane Head Mounted Displays Using 3D Gaze. Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces, Association for Computing Machinery.","DOI":"10.1145\/2598153.2598154"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"van der Meulen, H., Kun, A.L., and Shaer, O. (2017). What Are We Missing?. ISS \u201917: Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces, Association for Computing Machinery.","DOI":"10.1145\/3132272.3132278"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Mandryk, R., Hancock, M., Perry, M., and Cox, A. (2018). Pinpointing. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems\u2014CHI \u201918, ACM Press.","DOI":"10.1145\/3173574"},{"key":"ref_44","unstructured":"Barz, M., Kapp, S., Kuhn, J., and Sonntag, D. Automatic Recognition and Augmentation of Attended Objects in Real-time using Eye Tracking and a Head-mounted Display, Manuscript submitted for publication."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Cerrolaza, J.J., Villanueva, A., Villanueva, M., and Cabeza, R. (2012). Error characterization and compensation in eye tracking systems. Proceedings of the Symposium on Eye Tracking Research and Applications, ACM.","DOI":"10.1145\/2168556.2168595"},{"key":"ref_46","unstructured":"Microsoft (2020, November 25). Comfort. Available online: https:\/\/docs.microsoft.com\/de-de\/windows\/mixed-reality\/design\/comfort."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1912","DOI":"10.1109\/TVCG.2015.2473855","article-title":"Resolving the Vergence-Accommodation Conflict in Head-Mounted Displays","volume":"22","author":"Kramida","year":"2016","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Macinnes, J.J., Iqbal, S., Pearson, J., and Johnson, E.N. (2018). Wearable Eye-tracking for Research: Automated dynamic gaze mapping and accuracy\/precision comparisons across devices. bioRxiv.","DOI":"10.1101\/299925"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2234\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:39:41Z","timestamp":1760161181000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2234"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,23]]},"references-count":48,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["s21062234"],"URL":"https:\/\/doi.org\/10.3390\/s21062234","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,23]]}}}