{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T23:14:58Z","timestamp":1768346098591,"version":"3.49.0"},"reference-count":53,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2021,12,8]],"date-time":"2021-12-08T00:00:00Z","timestamp":1638921600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Statistical measurements of eye movement-specific properties, such as fixations, saccades, blinks, or pupil dilation, are frequently utilized as input features for machine learning algorithms applied to eye tracking recordings. These characteristics are intended to be interpretable aspects of eye gazing behavior. However, prior research has demonstrated that when trained on implicit representations of raw eye tracking data, neural networks outperform these traditional techniques. To leverage the strengths and information of both feature sets, we integrated implicit and explicit eye tracking features in one classification approach in this work. A neural network was adapted to process the heterogeneous input and predict the internally and externally directed attention of 154 participants. We compared the accuracies reached by the implicit and combined features for different window lengths and evaluated the approaches in terms of person- and task-independence. The results indicate that combining implicit and explicit feature extraction techniques for eye tracking data improves classification results for attentional state detection significantly. The attentional state was correctly classified during new tasks with an accuracy better than chance, and person-independent classification even outperformed person-dependently trained classifiers for some settings. For future experiments and applications that require eye tracking data classification, we suggest to consider implicit data representation in addition to interpretable explicit features.<\/jats:p>","DOI":"10.3390\/s21248205","type":"journal-article","created":{"date-parts":[[2021,12,8]],"date-time":"2021-12-08T23:30:00Z","timestamp":1639006200000},"page":"8205","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Combining Implicit and Explicit Feature Extraction for Eye Tracking: Attention Classification Using a Heterogeneous Input"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0601-0950","authenticated-orcid":false,"given":"Lisa-Marie","family":"Vortmann","sequence":"first","affiliation":[{"name":"Cognitive Systems Lab, Department of Mathematics and Computer Science, University of Bremen, 28359 Bremen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5203-8797","authenticated-orcid":false,"given":"Felix","family":"Putze","sequence":"additional","affiliation":[{"name":"Cognitive Systems Lab, Department of Mathematics and Computer Science, University of Bremen, 28359 Bremen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,8]]},"reference":[{"key":"ref_1","unstructured":"Huey, E.B. (1908). The Psychology and Pedagogy of Reading, The Macmillan Company."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1146\/annurev.psych.093008.100427","article-title":"A taxonomy of external and internal attention","volume":"62","author":"Chun","year":"2011","journal-title":"Annu. Rev. Psychol."},{"key":"ref_3","unstructured":"Conati, C., Merten, C., Amershi, S., and Muldner, K. (2007, January 22\u201326). Using eye-tracking data for high-level user modeling in adaptive interfaces. Proceedings of the AAAI, Vancouver, BC, Canada."},{"key":"ref_4","unstructured":"Merten, C., and Conati, C. (February, January 29). Eye-tracking to model and adapt to user meta-cognition in intelligent learning environments. Proceedings of the 11th International Conference on Intelligent User Interfaces, Sydney, Australia."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"10131","DOI":"10.1038\/s41598-021-89023-8","article-title":"Computer-aided autism diagnosis based on visual attention models using eye tracking","volume":"11","author":"Oliveira","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"16865","DOI":"10.1109\/ACCESS.2020.2966928","article-title":"Designing an adaptive interface: Using eye tracking to classify how information usage changes over time in partially automated vehicles","volume":"8","author":"Ulahannan","year":"2020","journal-title":"IEEE Access"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3338844","article-title":"Improving user experience of eye tracking-based interaction: Introspecting and adapting interfaces","volume":"26","author":"Menges","year":"2019","journal-title":"ACM Trans. Comput.-Hum. Interact."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Feit, A.M., Williams, S., Toledo, A., Paradiso, A., Kulkarni, H., Kane, S., and Morris, M.R. (2017, January 6\u201311). Toward everyday gaze input: Accuracy and precision of eye tracking and implications for design. Proceedings of the 2017 Chi Conference on Human Factors in Computing Systems, Denver, CO, USA.","DOI":"10.1145\/3025453.3025599"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Vortmann, L.M., and Putze, F. (2020, January 25\u201330). Attention-Aware Brain Computer Interface to Avoid Distractions in Augmented Reality. Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, CHI EA \u201920, Honolulu, HI, USA.","DOI":"10.1145\/3334480.3382889"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"625","DOI":"10.3389\/fnins.2021.664490","article-title":"Imaging Time Series of Eye Tracking Data to Classify Attentional States","volume":"15","author":"Vortmann","year":"2021","journal-title":"Front. Neurosci."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1007\/s11704-019-8208-z","article-title":"A survey on ensemble learning","volume":"14","author":"Dong","year":"2020","journal-title":"Front. Comput. Sci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"19581","DOI":"10.1109\/ACCESS.2017.2754299","article-title":"Webcam-based eye movement analysis using CNN","volume":"5","author":"Meng","year":"2017","journal-title":"IEEE Access"},{"key":"ref_13","unstructured":"Hutt, S., Hardey, J., Bixler, R., Stewart, A., Risko, E., and D\u2019Mello, S.K. (2017, January 25\u201328). Gaze-Based Detection of Mind Wandering during Lecture Viewing. Proceedings of the International Conference on Educational Data Mining (EDM), Wuhan, China."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Xuelin Huang, M., Li, J., Ngai, G., Leong, H.V., and Bulling, A. (2019, January 21\u201325). Moment-to-moment detection of internal thought from eye vergence behaviour. Proceedings of the 27th ACM International Conference on Multimedia (MM \u201919), Nice, France.","DOI":"10.1145\/3343031.3350573"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1092","DOI":"10.3389\/fpsyg.2017.01092","article-title":"Eye behavior associated with internally versus externally directed cognition","volume":"8","author":"Benedek","year":"2017","journal-title":"Front. Psychol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"e12977","DOI":"10.1111\/cogs.12977","article-title":"How Reliably Do Eye Parameters Indicate Internal Versus External Attentional Focus?","volume":"45","author":"Ceh","year":"2021","journal-title":"Cogn. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Elbattah, M., Gu\u00e9rin, J.L., Carette, R., Cilia, F., and Dequen, G. (2020, January 1\u20134). NLP-Based Approach to Detect Autism Spectrum Disorder in Saccadic Eye Movement. Proceedings of the 2020 IEEE Symposium Series on Computational Intelligence (SSCI), Canberra, Australia.","DOI":"10.1109\/SSCI47803.2020.9308238"},{"key":"ref_18","first-page":"181","article-title":"A review of feature extraction in sentiment analysis","volume":"4","author":"Asghar","year":"2014","journal-title":"J. Basic Appl. Sci. Res."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhang, Y., and Zhu, W. (2013, January 13\u201317). Extracting implicit features in online customer reviews for opinion mining. Proceedings of the 22nd International Conference on World Wide Web, Rio de Janeiro, Brazil.","DOI":"10.1145\/2487788.2487835"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zeng, L., and Li, F. (2013). A classification-based approach for implicit feature identification. Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data, Springer.","DOI":"10.1007\/978-3-642-41491-6_18"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Schouten, K., and Frasincar, F. (2014, January 18\u201320). Implicit feature extraction for sentiment analysis in consumer reviews. Proceedings of the International Conference on Applications of Natural Language to Data Bases\/Information Systems, Montpellier, France.","DOI":"10.1007\/978-3-319-07983-7_31"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Bandana, R. (2018, January 4\u20135). Sentiment analysis of movie reviews using heterogeneous features. Proceedings of the 2018 2nd International Conference on Electronics, Materials Engineering & Nano-Technology (IEMENTech), Science City, Kolkata, India.","DOI":"10.1109\/IEMENTECH.2018.8465346"},{"key":"ref_23","unstructured":"Wang, Z., and Oates, T. (2015, January 25\u201331). Imaging time-series to improve classification and imputation. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3463507","article-title":"Exploration of Person-Independent BCIs for Internal and External Attention-Detection in Augmented Reality","volume":"5","author":"Vortmann","year":"2021","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"134","DOI":"10.3758\/s13428-017-0857-y","article-title":"An automated behavioral measure of mind wandering during computerized reading","volume":"50","author":"Faber","year":"2018","journal-title":"Behav. Res. Methods"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1007\/s11257-015-9167-1","article-title":"Automatic gaze-based user-independent detection of mind wandering during computerized reading","volume":"26","author":"Bixler","year":"2016","journal-title":"User Model. User-Adapt. Interact."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1037\/a0030217","article-title":"Mind wandering in sentence reading: Decoupling the link between mind and eye","volume":"67","author":"Foulsham","year":"2013","journal-title":"Can. J. Exp. Psychol. Can. Psychol. Exp."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1037\/pag0000031","article-title":"Validating older adults\u2019 reports of less mind-wandering: An examination of eye movements and dispositional influences","volume":"30","author":"Frank","year":"2015","journal-title":"Psychol. Aging"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1814","DOI":"10.3758\/s13423-015-0845-0","article-title":"Sudden insight is associated with shutting out visual inputs","volume":"22","author":"Salvi","year":"2015","journal-title":"Psychon. Bull. Rev."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"601","DOI":"10.3758\/s13415-016-0417-4","article-title":"Pupillary correlates of lapses of sustained attention","volume":"16","author":"Unsworth","year":"2016","journal-title":"Cogn. Affect. Behav. Neurosci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3432","DOI":"10.3758\/s13414-020-02068-1","article-title":"Eye behavior predicts susceptibility to visual distraction during internally directed cognition","volume":"82","author":"Beaty","year":"2020","journal-title":"Atten. Percept. Psychophys."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Vortmann, L.M., Schult, M., Benedek, M., Walcher, S., and Putze, F. (2019, January 14\u201319). Real-Time Multimodal Classification of Internal and External Attention. Proceedings of the Adjunct of the 2019 International Conference on Multimodal Interaction, ICMI \u201919, Suzhou, China.","DOI":"10.1145\/3351529.3360658"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Li, J., Ngai, G., Leong, H.V., and Chan, S.C. (2016, January 10\u201314). Your eye tells how well you comprehend. Proceedings of the 2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC), Atlanta, GA, USA.","DOI":"10.1109\/COMPSAC.2016.220"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1353","DOI":"10.1016\/j.humov.2012.06.003","article-title":"The timing and temporal patterns of eye blinking are dynamically modulated by attention","volume":"31","author":"Oh","year":"2012","journal-title":"Hum. Mov. Sci."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2289","DOI":"10.1080\/17470218.2013.858170","article-title":"Window to the Wandering Mind: Pupillometry of Spontaneous Thought While Reading","volume":"66","author":"Franklin","year":"2013","journal-title":"Q. J. Exp. Psychol."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Pfleging, B., Fekety, D.K., Schmidt, A., and Kun, A.L. (2016, January 7\u201312). A model relating pupil diameter to mental workload and lighting conditions. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, CA, USA.","DOI":"10.1145\/2858036.2858117"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Toker, D., and Conati, C. (2017, January 9\u201312). Leveraging Pupil Dilation Measures for Understanding Users\u2019 Cognitive Load During Visualization Processing. Proceedings of the Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization, Bratislava, Slovakia.","DOI":"10.1145\/3099023.3099059"},{"key":"ref_38","unstructured":"Mills, C., Bixler, R., Wang, X., and D\u2019Mello, S.K. (July, January 29). Automatic Gaze-Based Detection of Mind Wandering during Narrative Film Comprehension. Proceedings of the International Conference on Educational Data Mining (EDM), Raleigh, NC, USA."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Puig, M.S., Zapata, L.P., Aznar-Casanova, J.A., and Sup\u00e8r, H. (2013). A role of eye vergence in covert attention. PLoS ONE, 8.","DOI":"10.1371\/journal.pone.0052955"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1177\/0956797613503556","article-title":"The eye pupil adjusts to imaginary light","volume":"25","author":"Laeng","year":"2014","journal-title":"Psychol. Sci."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"913","DOI":"10.3758\/s13428-013-0422-2","article-title":"PyGaze: An open-source, cross-platform toolbox for minimal-effort programming of eyetracking experiments","volume":"46","author":"Dalmaijer","year":"2014","journal-title":"Behav. Res. Methods"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Salvucci, D.D., and Goldberg, J.H. (2000, January 6\u20138). Identifying fixations and saccades in eye-tracking protocols. Proceedings of the 2000 Symposium on Eye Tracking Research & Applications, Palm Beach Gardens, FL, USA.","DOI":"10.1145\/355017.355028"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"881","DOI":"10.3758\/APP.71.4.881","article-title":"Fixation identification: The optimum threshold for a dispersion algorithm","volume":"71","author":"Blignaut","year":"2009","journal-title":"Atten. Percept. Psychophys."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Keogh, E.J., and Pazzani, M.J. (2000, January 20\u201323). Scaling up dynamic time warping for datamining applications. Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Boston, MA, USA.","DOI":"10.1145\/347090.347153"},{"key":"ref_45","first-page":"1","article-title":"pyts: A Python Package for Time Series Classification","volume":"21","author":"Faouzi","year":"2020","journal-title":"J. Mach. Learn. Res."},{"key":"ref_46","unstructured":"Wang, Z., and Oates, T. (2015, January 25\u201330). Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. Proceedings of the Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Yang, C.L., Chen, Z.X., and Yang, C.Y. (2020). Sensor Classification Using Convolutional Neural Network by Encoding Multivariate Time Series as Two-Dimensional Colored Images. Sensors, 20.","DOI":"10.3390\/s20010168"},{"key":"ref_48","first-page":"1929","article-title":"Dropout: A Simple Way to Prevent Neural Networks from Overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_49","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 6\u201311). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_50","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems 32, Curran Associates, Inc."},{"key":"ref_51","first-page":"52","article-title":"Better than random: A closer look on BCI results","volume":"10","author":"Scherer","year":"2008","journal-title":"Int. J. Bioelectromagn."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: A practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Jmour, N., Zayen, S., and Abdelkrim, A. (2018, January 22\u201325). Convolutional neural networks for image classification. Proceedings of the 2018 International Conference on Advanced Systems and Electric Technologies (IC_ASET), Hammamet, Tunisia.","DOI":"10.1109\/ASET.2018.8379889"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/24\/8205\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:43:24Z","timestamp":1760168604000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/24\/8205"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,8]]},"references-count":53,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["s21248205"],"URL":"https:\/\/doi.org\/10.3390\/s21248205","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,8]]}}}