{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T05:07:13Z","timestamp":1779253633520,"version":"3.51.4"},"reference-count":36,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T00:00:00Z","timestamp":1778284800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper proposes a model to predict users\u2019 personal interests from their gaze. Current image processing technologies enable us to identify each user\u2019s gaze and pupil diameter. However, they cannot identify user interests. The model uses deep learning technologies. It consists of mechanisms to handle the two kinds of gaze: one specific to individuals and the other common to many people. Its training method utilizes the human property that user interests in objects in vision affect their gaze. Since it is known that pupil diameters increase when users view what they are interested in, we can train the model to predict users\u2019 personal interests using the pupil diameters as labels. Notably, the paper not only proposes a model structure to predict personal interests but also examines the process of the modification of parameters of the model to reveal requisites to predict personal interests from gaze. The method enables us to provide personalized information to each user, such as recommendations in VoD services, advertisements during e-shopping, plate annotations in restaurant menus, and so on.<\/jats:p>","DOI":"10.3390\/s26102975","type":"journal-article","created":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T15:39:32Z","timestamp":1778513972000},"page":"2975","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Combining Deep Learning Mechanisms to Predict Interests from Gaze"],"prefix":"10.3390","volume":"26","author":[{"given":"Aoba","family":"Izumi","sequence":"first","affiliation":[{"name":"College of Information Science and Enginnering, Ritsumeikan University, Osaka 567-8570, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fumiko","family":"Harada","sequence":"additional","affiliation":[{"name":"College of Information Science and Enginnering, Ritsumeikan University, Osaka 567-8570, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4739-4292","authenticated-orcid":false,"given":"Hiromitsu","family":"Shimakawa","sequence":"additional","affiliation":[{"name":"College of Information Science and Enginnering, Ritsumeikan University, Osaka 567-8570, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,5,9]]},"reference":[{"key":"ref_1","first-page":"13","article-title":"The Netflix Recommender System: Algorithms, Business Value, and Innovation","volume":"6","author":"Hunt","year":"2016","journal-title":"ACM Trans. Manag. Inf. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1145\/345124.345169","article-title":"Automatic personalization based on Web usage mining","volume":"43","author":"Mobasher","year":"2000","journal-title":"Commun. ACM"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hong, L., and Davison, B.D. (2010). Empirical study of topic modeling in Twitter. Proceedings of the First Workshop on Social Media Analytics, SOMA \u201910, ACM.","DOI":"10.1145\/1964858.1964870"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1145\/3555373","article-title":"Personal or General? A Hybrid Strategy with Multi-factors for News Recommendation","volume":"41","author":"Huang","year":"2023","journal-title":"ACM Trans. Inf. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Srivastava, A. (2016). Gray Sheep, Influential Users, User Modeling and Recommender System Adoption by Startups. Proceedings of the 10th ACM Conference on Recommender Systems, RecSys \u201916, ACM.","DOI":"10.1145\/2959100.2959103"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Gras, B., Brun, A., and Boyer, A. (2016). Identifying Grey Sheep Users in Collaborative Filtering: A Distribution-Based Technique. Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization, UMAP \u201916, ACM.","DOI":"10.1145\/2930238.2930242"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"17043","DOI":"10.1007\/s00521-020-04920-9","article-title":"Neural embedding collaborative filtering for recommender systems","volume":"32","author":"Huang","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Sarvi, F., Aliannejadi, M., Schelter, S., and de Rijke, M. (2026). Understanding Visual Saliency of Outlier Items in Product Search. ACM Trans. Inf. Syst.","DOI":"10.1145\/3806228"},{"key":"ref_9","first-page":"157","article-title":"Revisited Visual Saliency Detection with Deep Learning: A Review of Recent Advancements","volume":"58","author":"Kumain","year":"2025","journal-title":"ACM Comput. Surv."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1470","DOI":"10.1587\/transinf.E93.D.1470","article-title":"Estimates of User Interest Using Timing Structures between Proactive Content-Display Updates and Eye Movements","volume":"93-D","author":"Hirayama","year":"2010","journal-title":"IEICE Trans."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1068\/i0382","article-title":"Yarbus, Eye Movements, and Vision","volume":"1","author":"Tatler","year":"2010","journal-title":"i-Perception"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Koide, N., Kubo, T., Nishida, S., Shibata, T., and Ikeda, K. (2015). Art expertise reduces influence of visual salience on fixation in viewing abstract-paintings. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0117696"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Jiang, Y., Leiva, L.A., Rezazadegan Tavakoli, H., Houssel, P.R.B., Kylm\u00e4l\u00e4, J., and Oulasvirta, A. (2023). UEyes: Understanding Visual Saliency across User Interface Types. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI \u201923, ACM.","DOI":"10.1145\/3544548.3581096"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1007\/s00415-012-6631-2","article-title":"High-throughput classification of clinical populations from natural viewing eye movements","volume":"260","author":"Tseng","year":"2013","journal-title":"J. Neurol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/BF01739737","article-title":"Attention and eye movement control: An overview","volume":"239","author":"Groner","year":"1989","journal-title":"Eur. Arch. Psychiatry Neurol. Sci."},{"key":"ref_16","first-page":"53","article-title":"The Eye in Extended Reality: A Survey on Gaze Interaction and Eye Tracking in Head-worn Extended Reality","volume":"55","author":"Plopski","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wenclik, L., and Touya, G. (2024). EyeCatchingMaps, a Dataset to Assess Saliency Models on Maps, Zenodo.","DOI":"10.5194\/agile-giss-5-51-2024"},{"key":"ref_18","unstructured":"Borji, A., and Itti, L. (2015). CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research. arXiv, Available online: https:\/\/arxiv.org\/abs\/1505.03581."},{"key":"ref_19","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv, Available online: http:\/\/arxiv.org\/abs\/1409.1556."},{"key":"ref_20","unstructured":"Xu, Y., Gao, S., Wu, J., Li, N., and Yu, J. (2017). Personalized Saliency and its Prediction. arXiv, Available online: http:\/\/arxiv.org\/abs\/1710.03011."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Strohm, F., B\u00e2ce, M., and Bulling, A. (2024). Learning User Embeddings from Human Gaze for Personalised Saliency Prediction. arXiv, Available online: http:\/\/arxiv.org\/abs\/2403.13653.","DOI":"10.1145\/3655603"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jiang, Y., Guo, Z., Tavakoli, H.R., Leiva, L.A., and Oulasvirta, A. (2024). EyeFormer: Predicting Personalized Scanpaths with Transformer-Guided Reinforcement Learning. arXiv, Available online: http:\/\/arxiv.org\/abs\/2404.10163.","DOI":"10.1145\/3654777.3676436"},{"key":"ref_23","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention Is All You Need. arXiv, Available online: http:\/\/arxiv.org\/abs\/1706.03762."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Tran, T., You, D., and Lee, K. (2020). Quaternion-Based Self-Attentive Long Short-term User Preference Encoding for Recommendation. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM \u201920, ACM.","DOI":"10.1145\/3340531.3411926"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wu, L., Li, S., Hsieh, C.J., and Sharpnack, J. (2020). SSE-PT: Sequential Recommendation Via Personalized Transformer. Proceedings of the 14th ACM Conference on Recommender Systems, RecSys \u201920, ACM.","DOI":"10.1145\/3383313.3412258"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Jiang, M., Huang, S., Duan, J., and Zhao, Q. (2015). SALICON: Saliency in Context. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.","DOI":"10.1109\/CVPR.2015.7298710"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Linardos, A., K\u00fcmmerer, M., Press, O., and Bethge, M. (2021). DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling. arXiv, Available online: http:\/\/arxiv.org\/abs\/2105.12441.","DOI":"10.1109\/ICCV48922.2021.01268"},{"key":"ref_28","unstructured":"Pan, J., Ferrer, C.C., McGuinness, K., O\u2019Connor, N.E., Torres, J., Sayrol, E., and Giro-i-Nieto, X. (2017). SalGAN: Visual Saliency Prediction with Generative Adversarial Networks. arXiv, Available online: http:\/\/arxiv.org\/abs\/1701.01081."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Doll\u00e1r, P. (2014). Microsoft COCO: Common Objects in Context. arXiv, Available online: http:\/\/arxiv.org\/abs\/1405.0312.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1037\/0096-3445.136.4.569","article-title":"Overcoming Intuition: Metacognitive Difficulty Activates Analytic Reasoning","volume":"136","author":"Alter","year":"2007","journal-title":"J. Exp. Psychol. Gen."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1167\/14.1.28","article-title":"Predicting human gaze beyond pixels","volume":"14","author":"Xu","year":"2014","journal-title":"J. Vis."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1126\/science.132.3423.349","article-title":"Pupil Size as Related to Interest Value of Visual Stimuli","volume":"132","author":"Hess","year":"1960","journal-title":"Science"},{"key":"ref_33","first-page":"382","article-title":"Few-shot Personalized Saliency Prediction Based on Interpersonal Gaze Patterns","volume":"13","author":"Moroto","year":"2025","journal-title":"ITE Trans. Media Technol. Appl."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Wang, Q., Outtas, M., Fournier, J., Etchamendy, E., Ch\u00e9rel, M., and Zhang, L. (2023). Predicting personalized saliency map for people with autism spectrum disorder. Proceedings of the 20th International Conference on Content-Based Multimedia Indexing, ACM.","DOI":"10.1145\/3617233.3617277"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1167\/9.8.447","article-title":"Saccades and microsaccades during visual fixation, exploration, and search: Foundations for a common saccadic generator","volume":"9","author":"Troncoso","year":"2010","journal-title":"J. Vis."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1320","DOI":"10.1007\/s00426-018-01141-7","article-title":"Microsaccadic rate and pupil size dynamics in pro-\/anti-saccade preparation: The impact of intermixed vs. blocked trial administration","volume":"84","author":"Dalmaso","year":"2020","journal-title":"Psychol. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/10\/2975\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T04:19:28Z","timestamp":1779250768000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/10\/2975"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,9]]},"references-count":36,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2026,5]]}},"alternative-id":["s26102975"],"URL":"https:\/\/doi.org\/10.3390\/s26102975","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,9]]}}}