{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T01:32:33Z","timestamp":1772847153827,"version":"3.50.1"},"reference-count":86,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2022,3,29]],"date-time":"2022-03-29T00:00:00Z","timestamp":1648512000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NSF","award":["CNS-1943509"],"award-info":[{"award-number":["CNS-1943509"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2022,3,29]]},"abstract":"<jats:p>As virtual reality (VR) offers an unprecedented experience than any existing multimedia technologies, VR videos, or called 360-degree videos, have attracted considerable attention from academia and industry. How to quantify and model end users' perceived quality in watching 360-degree videos, or called QoE, resides the center for high-quality provisioning of these multimedia services. In this work, we present EyeQoE, a novel QoE assessment model for 360-degree videos using ocular behaviors. Unlike prior approaches, which mostly rely on objective factors, EyeQoE leverages the new ocular sensing modality to comprehensively capture both subjective and objective impact factors for QoE modeling. We propose a novel method that models eye-based cues into graphs and develop a GCN-based classifier to produce QoE assessment by extracting intrinsic features from graph-structured data. We further exploit the Siamese network to eliminate the impact from subjects and visual stimuli heterogeneity. A domain adaptation scheme named MADA is also devised to generalize our model to a vast range of unseen 360-degree videos. Extensive tests are carried out with our collected dataset. Results show that EyeQoE achieves the best prediction accuracy at 92.9%, which outperforms state-of-the-art approaches. As another contribution of this work, we have publicized our dataset on https:\/\/github.com\/MobiSec-CSE-UTA\/EyeQoE_Dataset.git.<\/jats:p>","DOI":"10.1145\/3517240","type":"journal-article","created":{"date-parts":[[2022,3,29]],"date-time":"2022-03-29T13:42:46Z","timestamp":1648561366000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["EyeQoE"],"prefix":"10.1145","volume":"6","author":[{"given":"Huadi","family":"Zhu","sequence":"first","affiliation":[{"name":"The University of Texas at Arlington"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tianhao","family":"Li","sequence":"additional","affiliation":[{"name":"The University of Texas at Arlington"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chaowei","family":"Wang","sequence":"additional","affiliation":[{"name":"The University of Texas at Arlington"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenqiang","family":"Jin","sequence":"additional","affiliation":[{"name":"Hunan University and University of Texas at Arlington"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Srinivasan","family":"Murali","sequence":"additional","affiliation":[{"name":"The University of Texas at Arlington"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mingyan","family":"Xiao","sequence":"additional","affiliation":[{"name":"The University of Texas at Arlington"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dongqing","family":"Ye","sequence":"additional","affiliation":[{"name":"The University of Texas at Arlington"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ming","family":"Li","sequence":"additional","affiliation":[{"name":"The University of Texas at Arlington"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,3,29]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-1-55860-659-3.50030-5"},{"key":"e_1_2_1_2_1","unstructured":"2021. High-resolution VR Headset for Professionals - Varjo VR-3. https:\/\/varjo.com\/products\/vr-3\/"},{"key":"e_1_2_1_3_1","unstructured":"2021. HTC VIVE Pro Eye. https:\/\/www.vive.com\/eu\/product\/vive-pro-eye\/overview\/"},{"key":"e_1_2_1_4_1","unstructured":"2021. Neo 2 Neo 2 Eye: All-in-One VR Headset with 6DoF tracking. https:\/\/www.pico-interactive.com\/us\/neo2.html"},{"key":"e_1_2_1_5_1","unstructured":"2021. Tobii VR: Eye Tracking Technology in Virtual Reality. https:\/\/vr.tobii.com\/"},{"key":"e_1_2_1_6_1","unstructured":"2021. VR Eye Tracking For Business: FOVE 0 EYE TRACKING VR DEVKIT. https:\/\/fove-inc.com\/"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2936470"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2017.2705423"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2017.01092"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1002\/mpr.1833"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2807452"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/QoMEX.2018.8463422"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP39728.2021.9413783"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICME.2018.8486584"},{"key":"e_1_2_1_15_1","volume-title":"Yu-Chiang Frank Wang, and Jia-Bin Huang","author":"Chen Wei-Yu","year":"2019","unstructured":"Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. 2019. A Closer Look at Few-shot Classification."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV45572.2020.9093511"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2005.202"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.16910\/jemr.12.13"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","unstructured":"Gabriela Csurka. 2017. Domain Adaptation for Visual Applications: A Comprehensive Survey. https:\/\/doi.org\/10.1007\/978-3-319-58347-1_1","DOI":"10.1007\/978-3-319-58347-1_1"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3204949.3204966"},{"key":"e_1_2_1_21_1","unstructured":"Micha\u00ebl Defferrard Xavier Bresson and Pierre Vandergheynst. 2017. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. arXiv:1606.09375 [cs.LG]"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.3390\/fi11080171"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-57883-5"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/QoMEX.2016.7498964"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/QOMEX.2010.5516159"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2019.2956631"},{"key":"e_1_2_1_27_1","unstructured":"International Organization for Standardization. 2005. Image Safety - Reducing the Incidence of Undesirable Biomedical Effects Caused by Visual Image Sequences. ISO. https:\/\/books.google.com\/books?id=LfAncgAACAAJ"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","unstructured":"Chiara Galdi and Michele Nappi. 2019. Eye Movement Analysis in Biometrics. 171--183. https:\/\/doi.org\/10.1007\/978-981-13-1144-4_8","DOI":"10.1007\/978-981-13-1144-4_8"},{"key":"e_1_2_1_29_1","first-page":"1","article-title":"Domain-Adversarial Training of Neural Networks","volume":"17","author":"Ganin Yaroslav","year":"2016","unstructured":"Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Fran\u00e7ois Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-Adversarial Training of Neural Networks. The Journal of Machine Learning Research 17, 1 (Jan. 2016), 2096--2030.","journal-title":"The Journal of Machine Learning Research"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/MMSP.2019.8901743"},{"key":"e_1_2_1_31_1","doi-asserted-by":"crossref","unstructured":"E. H. Hess. 1975. The role of pupil size in communication. (1975) 110--119.","DOI":"10.1038\/scientificamerican1175-110"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1587\/transinf.2017MUP0011"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIHT.2017.7899148"},{"key":"e_1_2_1_34_1","unstructured":"ITU-T. 2007. Definition of Quality of Experience (QoE). https:\/\/www.itu.int\/md\/T05-FG.IPTV-IL-0050\/en"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2745197.2745202"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","unstructured":"Conor Keighrey Ronan Flynn Siobhan Murray and Niall Murray. 2017. A QoE Evaluation of Immersive Augmented and Virtual Reality Speech Language Assessment Applications. https:\/\/doi.org\/10.1109\/QoMEX.2017.7965656","DOI":"10.1109\/QoMEX.2017.7965656"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDSP.2011.6004999"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2880509"},{"key":"e_1_2_1_39_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2017","unstructured":"Diederik P. Kingma and Jimmy Ba. 2017. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs.LG]"},{"key":"e_1_2_1_40_1","volume-title":"Kipf and Max Welling","author":"Thomas","year":"2016","unstructured":"Thomas N. Kipf and Max Welling. 2016. Semi-Supervised Classification with Graph Convolutional Networks. arXiv e-prints, Article arXiv:1609.02907 (Sept. 2016), arXiv:1609.02907 pages. arXiv:1609.02907 [cs.LG]"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2008.4712319"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.visres.2007.06.015"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3240508.3240581"},{"key":"e_1_2_1_44_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Li Chen","year":"2019","unstructured":"Chen Li, Mai Xu, Lai Jiang, Shanyi Zhang, and Xiaoming Tao. 2019. Viewport Proposal CNN for 360deg Video Quality Assessment. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_1_45_1","unstructured":"Yujia Li Daniel Tarlow Marc Brockschmidt and Richard Zemel. 2017. Gated Graph Sequence Neural Networks. arXiv:1511.05493 [cs.LG]"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2011.2133770"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3303080"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1076\/0271-3683(200007)2111-zft535"},{"key":"e_1_2_1_49_1","volume-title":"Multi-Modal Domain Adaptation for Fine-Grained Action Recognition. CoRR abs\/2001.09691","author":"Munro Jonathan","year":"2020","unstructured":"Jonathan Munro and Dima Damen. 2020. Multi-Modal Domain Adaptation for Fine-Grained Action Recognition. CoRR abs\/2001.09691 (2020). arXiv:2001.09691 https:\/\/arxiv.org\/abs\/2001.09691"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-1-78548-109-3.50005-9"},{"key":"e_1_2_1_51_1","unstructured":"Jonny O'Dwyer Niall Murray and Ronan Flynn. 2020. Eye-based Continuous Affect Prediction. arXiv:1907.09896 [cs.HC]"},{"key":"e_1_2_1_52_1","unstructured":"Sean O'Kane. 2021. Tesla starts using in-car camera for Autopilot driver monitoring - The Verge. https:\/\/www.theverge.com\/2021\/5\/27\/22457430\/tesla-in-car-camera-driver-monitoring-system"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1996.8.5.895"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1117\/12.675760"},{"key":"e_1_2_1_55_1","volume-title":"Domain Adaptation in Sentiment Analysis of Twitter (AAAIWS'11-05)","author":"Kiran Peddinti Viswa Mani","unstructured":"Viswa Mani Kiran Peddinti and Prakriti Chintalapoodi. 2011. Domain Adaptation in Sentiment Analysis of Twitter (AAAIWS'11-05). AAAI Press, 44--49."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCAS.2006.1688199"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNSM.2020.3018303"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-1-78548-236-6.50002-7"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-009-9124-7"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-22127-w"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3204949.3208118"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2008.2005605"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2009.2034992"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","unstructured":"Ashutosh Singla Stephan Fremerey Werner Robitza Pierre Lebreton and Alexander Raake. 2017. Comparison of Subjective Quality Evaluation for HEVC Encoded Omnidirectional Videos at Different Bit-rates for UHD and FHD Resolution. 511--519. https:\/\/doi.org\/10.1145\/3126686.3126768","DOI":"10.1145\/3126686.3126768"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/2976749.2978311"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISCAS.2019.8702664"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2017.2720693"},{"key":"e_1_2_1_68_1","volume-title":"Proceedings of the 27th International Conference on Neural Information Processing Systems -","volume":"2","author":"Sutskever Ilya","unstructured":"Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 (Montreal, Canada) (NIPS'14). MIT Press, Cambridge, MA, USA, 3104--3112."},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-00958-7_31"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1109\/QoMEX.2018.8463414"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/MMSP.2017.8122249"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJM199302253280817"},{"key":"e_1_2_1_73_1","volume-title":"Few-shot Learning: A Survey. CoRR abs\/1904.05046","author":"Wang Yaqing","year":"2019","unstructured":"Yaqing Wang and Quanming Yao. 2019. Few-shot Learning: A Survey. CoRR abs\/1904.05046 (2019). arXiv:1904.05046 http:\/\/arxiv.org\/abs\/1904.05046"},{"key":"e_1_2_1_74_1","volume-title":"An evaluation framework for 360-degree video compression. 2017 IEEE Visual Communications and Image Processing (VCIP)","author":"Xiu Xiaoyu","year":"2017","unstructured":"Xiaoyu Xiu, Yuwen He, Yan Ye, and Bharath Vishwanath. 2017. An evaluation framework for 360-degree video compression. 2017 IEEE Visual Communications and Image Processing (VCIP) (2017), 1--4."},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2018.2886277"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICMEW.2014.6890604"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1109\/COMST.2020.3006999"},{"key":"e_1_2_1_78_1","volume-title":"Meta-learning for Few-shot Natural Language Processing: A Survey. CoRR abs\/2007.09604","author":"Yin Wenpeng","year":"2020","unstructured":"Wenpeng Yin. 2020. Meta-learning for Few-shot Natural Language Processing: A Survey. CoRR abs\/2007.09604 (2020). arXiv:2007.09604https:\/\/arxiv.org\/abs\/2007.09604"},{"key":"e_1_2_1_79_1","volume-title":"Diverse Few-Shot Text Classification with Multiple Metrics. CoRR abs\/1805.07513","author":"Yu Mo","year":"2018","unstructured":"Mo Yu, Xiaoxiao Guo, Jinfeng Yi, Shiyu Chang, Saloni Potdar, Yu Cheng, Gerald Tesauro, Haoyu Wang, and Bowen Zhou. 2018. Diverse Few-Shot Text Classification with Multiple Metrics. CoRR abs\/1805.07513 (2018). arXiv:1805.07513 http:\/\/arxiv.org\/abs\/1805.07513"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2015.12"},{"key":"e_1_2_1_81_1","doi-asserted-by":"crossref","unstructured":"Vladyslav Zakharchenko K. Choi and J. Park. 2016. Quality metric for spherical panoramic video. In Optical Engineering + Applications.","DOI":"10.1117\/12.2235885"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-31577-1"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-31577-1"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1186\/s40649-019-0069-y"},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSP.2018.8652269"},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2864872"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3517240","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3517240","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3517240","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T04:25:22Z","timestamp":1752467122000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3517240"}},"subtitle":["A Novel QoE Assessment Model for 360-degree Videos Using Ocular Behaviors"],"short-title":[],"issued":{"date-parts":[[2022,3,29]]},"references-count":86,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,3,29]]}},"alternative-id":["10.1145\/3517240"],"URL":"https:\/\/doi.org\/10.1145\/3517240","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,29]]},"assertion":[{"value":"2022-03-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}