{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,20]],"date-time":"2025-11-20T06:50:38Z","timestamp":1763621438549,"version":"3.41.0"},"reference-count":41,"publisher":"Association for Computing Machinery (ACM)","issue":"ETRA","license":[{"start":{"date-parts":[[2023,5,17]],"date-time":"2023-05-17T00:00:00Z","timestamp":1684281600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Health Research BC"},{"name":"The Canadian Consortium of Neurodegeneration and Aging"},{"name":"The Alzheimer's Society of Canada"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2023,5,17]]},"abstract":"<jats:p>There has been increasing interest in non-invasive predictors of Alzheimer's disease (AD) as an initial screen for this condition. Previously, successful attempts leveraged eye-tracking and language data generated during picture narration and reading tasks. These results were obtained with high-end, expensive eye-trackers. Instead, we explore classification using eye-tracking data collected with a webcam, where our classifiers are built using a deep-learning approach. Our results show that the webcam gaze classifier is not as good as the classifier based on high-end eye-tracking data. However, the webcam-based classifier still beats the majority-class baseline classifier in terms of AU-ROC, indicating that predictive signals can be extracted from webcam gaze tracking. Hence, although our results indicate that there is still a long way to go before webcam gaze tracking can reach practical relevance, they still provide an encouraging proof of concept that this technology should be further explored as an affordable alternative to high-end eye-trackers for the detection of AD.<\/jats:p>","DOI":"10.1145\/3591126","type":"journal-article","created":{"date-parts":[[2023,5,18]],"date-time":"2023-05-18T20:21:03Z","timestamp":1684441263000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Classification of Alzheimer's using Deep-learning Methods on Webcam-based Gaze Data"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7206-4888","authenticated-orcid":false,"given":"Anuj","family":"Harisinghani","sequence":"first","affiliation":[{"name":"The University of British Columbia, Vancouver, BC, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8185-3475","authenticated-orcid":false,"given":"Harshinee","family":"Sriram","sequence":"additional","affiliation":[{"name":"The University of British Columbia, Vancouver, BC, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8434-9335","authenticated-orcid":false,"given":"Cristina","family":"Conati","sequence":"additional","affiliation":[{"name":"The University of British Columbia, Vancouver, BC, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4310-0119","authenticated-orcid":false,"given":"Giuseppe","family":"Carenini","sequence":"additional","affiliation":[{"name":"The University of British Columbia, Vancouver, BC, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1176-0633","authenticated-orcid":false,"given":"Thalia","family":"Field","sequence":"additional","affiliation":[{"name":"The University of British Columbia, Vancouver, BC, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7652-1158","authenticated-orcid":false,"given":"Hyeju","family":"Jang","sequence":"additional","affiliation":[{"name":"The University of British Columbia, Vancouver, BC, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2233-6288","authenticated-orcid":false,"given":"Gabriel","family":"Murray","sequence":"additional","affiliation":[{"name":"University of the Fraser Valley, Abbotsford, BC, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,5,18]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Yao Chong Lim, and Louis-Philippe Morency","author":"Zadeh Amir","year":"2018","unstructured":"Baltrusaitis, Tadas, Amir Zadeh, Yao Chong Lim, and Louis-Philippe Morency. 2018. \"OpenFace 2.0: Facial Behavior Analysis Toolkit.\" Pp. 59--66 in 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1474--4422(11)70072--2"},{"volume-title":"813--41 in Proceedings of the 5th Machine Learning for Healthcare Conference. PMLR","author":"Jang Hyeju","key":"e_1_2_2_3_1","unstructured":"Barral, Oswald, Hyeju Jang, Sally Newton-Mason, Sheetal Shajan, Thomas Soroski, Giuseppe Carenini, Cristina Conati, and Thalia Field. 2020. \"Non-Invasive Classification of Alzheimer's Disease Using Eye Tracking and Language.\" Pp. 813--41 in Proceedings of the 5th Machine Learning for Healthcare Conference. PMLR."},{"key":"e_1_2_2_4_1","unstructured":"Biondi Juan Gerardo Fernandez Silvia Castro and Osvaldo Agamennoni. 2018. \"Eye-Movement Behavior Identification for AD Diagnosis.\""},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-015--9167--1"},{"key":"e_1_2_2_6_1","series-title":"Lecture Notes in Computer Science","volume-title":"98--103 in Artificial Intelligence in Education","author":"Agarwal Mansi","unstructured":"Boote, Bikram, Mansi Agarwal, and Jack Mostow. 2021. \"Early Prediction of Children's Disengagement in a Tablet Tutor Using Visual Features.\" Pp. 98--103 in Artificial Intelligence in Education, Lecture Notes in Computer Science, edited by I. Roll, D. McNamara, S. Sosnovsky, R. Luckin, and V. Dimitrova. Cham: Springer International Publishing."},{"key":"e_1_2_2_7_1","doi-asserted-by":"crossref","unstructured":"Cho Kyunghyun Bart van Merrienboer Dzmitry Bahdanau and Yoshua Bengio. 2014. \"On the Properties of Neural Machine Translation: Encoder-Decoder Approaches.\"","DOI":"10.3115\/v1\/W14-4012"},{"key":"e_1_2_2_8_1","unstructured":"Chung Junyoung Caglar Gulcehre KyungHyun Cho and Yoshua Bengio. 2014. \"Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling.\""},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jalz.2012.09.011"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1075\/ps.17011.cum"},{"volume-title":"Zone out No More: Mitigating Mind Wandering during Computerized Reading","author":"Sidney","key":"e_1_2_2_11_1","unstructured":"D'Mello, Sidney K., Caitlin Mills, Robert Bixler, and Nigel Bosch. 2017. Zone out No More: Mitigating Mind Wandering during Computerized Reading. International Educational Data Mining Society."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1080\/13803395.2014.892060"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.3233\/JAD-150520"},{"volume-title":"243--54 in Proceedings of the 2nd Machine Learning for Healthcare Conference. PMLR","author":"Hariharan Sanjay","key":"e_1_2_2_14_1","unstructured":"Futoma, Joseph, Sanjay Hariharan, Katherine Heller, Mark Sendak, Nathan Brajer, Meredith Clement, Armando Bedoya, and Cara O'Brien. 2017. \"An Improved Multi-Output Gaussian Process RNN with Real-Time Validation for Early Sepsis Detection.\" Pp. 243--54 in Proceedings of the 2nd Machine Learning for Healthcare Conference. PMLR."},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1093\/brain"},{"key":"e_1_2_2_16_1","unstructured":"Goodglass Harold and Edith Kaplan. 1972. The Assessment of Aphasia and Related Disorders. Lea & Febiger."},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","unstructured":"Grnarova Paulina Florian Schmidt Stephanie L. Hyland and Carsten Eickhoff. 2016. \"Neural Document Embeddings for Intensive Care Patient Mortality Prediction.\" doi: 10.48550\/arXiv.1612.00467.","DOI":"10.48550\/arXiv.1612.00467"},{"volume-title":"ICMI '22","author":"Sidney","key":"e_1_2_2_18_1","unstructured":"Hutt, Stephen, and Sidney K. D'Mello. 2022. \"Evaluating Calibration-Free Webcam-Based Eye Tracking for Gaze-Based User Modeling.\" Pp. 224--35 in Proceedings of the 2022 International Conference on Multimodal Interaction, ICMI '22. New York, NY, USA: Association for Computing Machinery."},{"key":"e_1_2_2_19_1","volume-title":"Caitlin Lewis, and Pavan Tutt.","author":"Soroski Thomas","year":"2021","unstructured":"Jang, Hyeju, Thomas Soroski, Matteo Rizzo, Oswald Barral, Anuj Harisinghani, Sally Newton-Mason, Saffrin Granby, Thiago Monnerat Stutz da Cunha Vasco, Caitlin Lewis, and Pavan Tutt. 2021. \"Classification of Alzheimer's Disease Leveraging Multi-Task Machine Learning Analysis of Speech and Eye-Movement Data.\" Frontiers in Human Neuroscience 512."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2017.2735633"},{"key":"e_1_2_2_21_1","doi-asserted-by":"crossref","unstructured":"Karlekar Sweta Tong Niu and Mohit Bansal. 2018. \"Detecting Linguistic Characteristics of Alzheimer's Dementia by Interpreting Neural Models.\"","DOI":"10.18653\/v1\/N18-2110"},{"volume-title":"270--86 in Proceedings of the 4th Machine Learning for Healthcare Conference. PMLR","author":"Jang Hyeju","key":"e_1_2_2_22_1","unstructured":"Kong, Weirui, Hyeju Jang, Giuseppe Carenini, and Thalia Field. 2019. \"A Neural Model for Predicting Dementia from Language.\" Pp. 270--86 in Proceedings of the 4th Machine Learning for Healthcare Conference. PMLR."},{"key":"e_1_2_2_23_1","doi-asserted-by":"crossref","unstructured":"Krafka Kyle Aditya Khosla Petr Kellnhofer Harini Kannan Suchendra Bhandarkar Wojciech Matusik and Antonio Torralba. 2016. \"Eye Tracking for Everyone.\" Pp. 2176--84 in.","DOI":"10.1109\/CVPR.2016.239"},{"key":"e_1_2_2_24_1","unstructured":"Lall\u00e9 S\u00e9bastien Cristina Conati and Giuseppe Carenini. 2016. \"Predicting Confusion in Information Visualization from Eye Tracking and Interaction Data.\" Pp. 2529--35 in IJCAI."},{"key":"e_1_2_2_25_1","doi-asserted-by":"crossref","unstructured":"Li Shuai Wanqing Li Chris Cook Ce Zhu and Yanbo Gao. 2018. \"Independently Recurrent Neural Network (Indrnn): Building a Longer and Deeper Rnn.\" Pp. 5457--66 in Proceedings of the IEEE conference on computer vision and pattern recognition.","DOI":"10.1109\/CVPR.2018.00572"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","unstructured":"Lipton Zachary C. David C. Kale Charles Elkan and Randall Wetzel. 2015. \"Learning to Diagnose with LSTM Recurrent Neural Networks.\" doi: 10.48550\/arXiv.1511.03677.","DOI":"10.48550\/arXiv.1511.03677"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1097\/WCO.0000000000000274"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.3233\/JAD-141173"},{"key":"e_1_2_2_29_1","volume-title":"ETRA '18","author":"Huang Michael Xuelin","year":"2018","unstructured":"M\u00fcller, Philipp, Michael Xuelin Huang, Xucong Zhang, and Andreas Bulling. 2018. \"Robust Eye Contact Detection in Natural Multi-Person Interactions Using Gaze and Speaking Behaviour.\" Pp. 1--10 in Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications, ETRA '18. New York, NY, USA: Association for Computing Machinery."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1532--5415.2005.53221.x"},{"key":"e_1_2_2_31_1","doi-asserted-by":"crossref","unstructured":"Pavisic Ivanna M. Nicholas C. Firth Samuel Parsons David Martinez Rego Timothy J. Shakespeare Keir X. X. Yong Catherine F. Slattery Ross W. Paterson Alexander J. M. Foulkes Kirsty Macpherson Amelia M. Carton Daniel C. Alexander John Shawe-Taylor Nick C. Fox Jonathan M. Schott Sebastian J. Crutch and Silvia Primativo. 2017. \"Eyetracking Metrics in Young Onset Alzheimer's Disease: A Window into Cognitive Visual Functions.\" Frontiers in Neurology 8.","DOI":"10.3389\/fneur.2017.00377"},{"key":"e_1_2_2_32_1","unstructured":"Prince Martin Dr Renata Bryce and Dr Cleusa Ferri. n.d. \"World Alzheimer Report 2011: The Benefits of Early Diagnosis and Intervention.\""},{"key":"e_1_2_2_33_1","series-title":"Lecture Notes in Computer Science","volume-title":"317--25 in Medical Image Computing and Computer-Assisted Intervention -- MICCAI","author":"Esteva Andre","year":"2016","unstructured":"Pusiol, Guido, Andre Esteva, Scott S. Hall, Michael Frank, Arnold Milstein, and Li Fei-Fei. 2016. \"Vision-Based Classification of Developmental Disorders Using Eye-Movements.\" Pp. 317--25 in Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2016, Lecture Notes in Computer Science, edited by S. Ourselin, L. Joskowicz, M. R. Sabuncu, G. Unal, and W. Wells. Cham: Springer International Publishing."},{"key":"e_1_2_2_34_1","volume-title":"ICMI '20","author":"Shane","year":"2020","unstructured":"Sims, Shane D., and Cristina Conati. 2020. \"A Neural Architecture for Detecting User Confusion in Eye-Tracking Data.\" Pp. 15--23 in Proceedings of the 2020 International Conference on Multimodal Interaction, ICMI '20. New York, NY, USA: Association for Computing Machinery."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","unstructured":"Tay Yi Mostafa Dehghani Dara Bahri and Donald Metzler. 2022. \"Efficient Transformers: A Survey.\" ACM Computing Surveys. doi: 10.1145\/3530811.","DOI":"10.1145\/3530811"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2020.2979440"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1167\/iovs.11--8284"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","unstructured":"Tripathi Subarna Zachary C. Lipton Serge Belongie and Truong Nguyen. 2016. \"Context Matters: Refining Object Detection in Video with Recurrent Neural Networks.\" doi: 10.48550\/arXiv.1607.04648.","DOI":"10.48550\/arXiv.1607.04648"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020--18360--5"},{"key":"e_1_2_2_40_1","doi-asserted-by":"crossref","unstructured":"Yue-Hei Ng Joe Matthew Hausknecht Sudheendra Vijayanarasimhan Oriol Vinyals Rajat Monga and George Toderici. 2015. \"Beyond Short Snippets: Deep Networks for Video Classification.\" Pp. 4694--4702 in.","DOI":"10.1109\/CVPR.2015.7299101"},{"volume-title":"193--203 in Proceedings of the 30th annual ACM symposium on user interface software and technology","author":"Sugano Yusuke","key":"e_1_2_2_41_1","unstructured":"Zhang, Xucong, Yusuke Sugano, and Andreas Bulling. 2017. \"Everyday Eye Contact Detection Using Unsupervised Gaze Target Discovery.\" Pp. 193--203 in Proceedings of the 30th annual ACM symposium on user interface software and technology."}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3591126","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3591126","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:30Z","timestamp":1750178250000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3591126"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,17]]},"references-count":41,"journal-issue":{"issue":"ETRA","published-print":{"date-parts":[[2023,5,17]]}},"alternative-id":["10.1145\/3591126"],"URL":"https:\/\/doi.org\/10.1145\/3591126","relation":{},"ISSN":["2573-0142"],"issn-type":[{"type":"electronic","value":"2573-0142"}],"subject":[],"published":{"date-parts":[[2023,5,17]]},"assertion":[{"value":"2023-05-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}