{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T15:09:00Z","timestamp":1789744140694,"version":"4.0.1"},"reference-count":41,"publisher":"Association for Computing Machinery (ACM)","issue":"5","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Sen. Netw."],"published-print":{"date-parts":[[2026,9,30]]},"abstract":"<jats:p>The implantation of advertisement (ad) within VR videos presents a meaningful opportunity to engage users in immersive and interactive ways, enhancing the potential for effective advertising. However, the challenge lies in seamlessly implanting ad while maintaining the user\u2019s immersive experience. To solve this problem, we construct a dataset comprising VR images and videos with eye tracking data, providing a crucial resource for gaining in-depth insights into user interactions within virtual environments. Based on this dataset, we propose a reinforcement learning model called RL-Ad, built on a Deep Q-Network (DQN), which can dynamically adjust ad implantation regions according to users\u2019 eye movements. The proposed RL-Ad model aims at more precisely capturing user attention and increase their viewing duration on the implanted ads. Experimental results demonstrate that the proposed model can significantly improve the ad implantation by optimizing ad position and size to capture user attention while avoiding the interference in user experience.<\/jats:p>","DOI":"10.1145\/3744647","type":"journal-article","created":{"date-parts":[[2025,6,16]],"date-time":"2025-06-16T07:16:59Z","timestamp":1750058219000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["RL-Ad: Integrating Eye-Tracking into a Reinforcement Learning Model for VR Video Ad Implantation"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-4824-3158","authenticated-orcid":false,"given":"Qian","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Computer and Artificial Intelligence, Beijing Technology and Business University","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8166-6317","authenticated-orcid":false,"given":"Yahui","family":"Tang","sequence":"additional","affiliation":[{"name":"School of Computer and Artificial Intelligence, Beijing Technology and Business University","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4334-6103","authenticated-orcid":false,"given":"Chen","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer and Artificial Intelligence, Beijing Technology and Business University","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5502-2974","authenticated-orcid":false,"given":"Xun","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer and Artificial Intelligence, Beijing Technology and Business University","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7503-3021","authenticated-orcid":false,"given":"Xiaoming","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Computer and Artificial Intelligence, Beijing Technology and Business University","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3158-9650","authenticated-orcid":false,"given":"Vera","family":"Chung","sequence":"additional","affiliation":[{"name":"School of Computer Science, The University of Sydney","place":["Sydney, Australia"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,9,18]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"crossref","DOI":"10.1007\/s10055-023-00903-y","article-title":"Eye-tracking on virtual reality: A survey","author":"Moreno-Arjonilla Jes\u00fas","year":"2024","unstructured":"Jes\u00fas Moreno-Arjonilla, Alfonso L\u00f3pez-Ruiz, J. Roberto Jim\u00e9nez-P\u00e9rez, Jos\u00e9 E. Callejas-Aguilera, and Juan M. Jurado. 2024. Eye-tracking on virtual reality: A survey. Virtual Reality 28, 1 (2024), 38.","journal-title":"Virtual Reality 28"},{"key":"e_1_3_1_3_2","doi-asserted-by":"crossref","unstructured":"Andreas Riegler Andreas Riener Philipp Wintersberger Tamara Von Sawitzky and Ye Eun Song. 2022. Workshop on automotive mixed reality applications: Transitional interfaces multi-user VR and helmet-mounted AR for cyclists. In Adjunct Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. 172\u2013175.","DOI":"10.1145\/3544999.3550158"},{"key":"e_1_3_1_4_2","doi-asserted-by":"crossref","unstructured":"Steven Jiang Lawrence Lim and Misha Sra. 2023. Spatializing music in virtual reality. In Proceedings of the 2023 ACM Symposium on Spatial User Interaction (SUI\u201923). Association for Computing Machinery New York NY USA Article 42 1\u20133.","DOI":"10.1145\/3607822.3618011"},{"key":"e_1_3_1_5_2","doi-asserted-by":"crossref","unstructured":"A. Shameem KK Ramachandran Amod Sharma Rajesh Singh Franklin John Selvaraj and Geetha Manoharan. 2023. The rising importance of ai in boosting the efficiency of online advertising in developing countries. In 2023 3rd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE). IEEE 1762\u20131766.","DOI":"10.1109\/ICACITE57410.2023.10182754"},{"key":"e_1_3_1_6_2","doi-asserted-by":"crossref","first-page":"3181","DOI":"10.1007\/s10055-023-00864-2","article-title":"Evaluating the viewer experience of interactive virtual reality movies","author":"Carpio Rudy","year":"2023","unstructured":"Rudy Carpio, Oliver Baumann, and James Birt. 2023. Evaluating the viewer experience of interactive virtual reality movies. Virtual Reality 27, 4 (2023), 3181\u20133190.","journal-title":"Virtual Reality 27"},{"key":"e_1_3_1_7_2","article-title":"Automatic detection of optimal ad insertion points in video","author":"Chen Ji","year":"2016","unstructured":"Ji Chen and Yu Chen. 2016. Automatic detection of optimal ad insertion points in video. Technical Disclosure Commons (2016).","journal-title":"Technical Disclosure Commons"},{"key":"e_1_3_1_8_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-27645-3","article-title":"Reinforcement learning","author":"Wiering Marco A.","year":"2012","unstructured":"Marco A. Wiering and Martijn Van Otterlo. 2012. Reinforcement learning. Adaptation, Learning, and Optimization 12, 3 (2012), 729.","journal-title":"Adaptation, Learning, and Optimization"},{"issue":"4","key":"e_1_3_1_9_2","first-page":"996","article-title":"Visual saliency based on scale-space analysis in the frequency domain","volume":"35","author":"Li Jian","year":"2012","unstructured":"Jian Li, Martin D. Levine, Xiangjing An, Xin Xu, and Hangen He. 2012. Visual saliency based on scale-space analysis in the frequency domain. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 4 (2012), 996\u20131010.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_1_10_2","unstructured":"Jianqing Fan Zhaoran Wang Yuchen Xie and Zhuoran Yang. 2020. A theoretical analysis of deep q-learning. In Learning for Dynamics and Control. PMLR 486\u2013489."},{"key":"e_1_3_1_11_2","doi-asserted-by":"crossref","unstructured":"Hado Van Hasselt Arthur Guez and David Silver. 2016. Deep reinforcement learning with double q-learning. In Proceedings of the AAAI Conference on Artificial Intelligence 30 (2016).","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"e_1_3_1_12_2","doi-asserted-by":"crossref","first-page":"2723","DOI":"10.1007\/s10055-023-00839-3","article-title":"Passive identification of subjective preferences toward individual items using eye-tracking in a virtual reality environment","author":"Gabay Michal","year":"2023","unstructured":"Michal Gabay and Tom Schonberg. 2023. Passive identification of subjective preferences toward individual items using eye-tracking in a virtual reality environment. Virtual Reality 27, 3 (2023), 2723\u20132743.","journal-title":"Virtual Reality 27"},{"key":"e_1_3_1_13_2","doi-asserted-by":"crossref","unstructured":"Yahui Tang Chen Wang Hui Ye Xiaoming Chen and Vera Chung. 2024. SISAd: Advertisement implantation in VR video with saliency-constrained instance segmentation. In 2024 IEEE International Conference on Consumer Electronics (ICCE). IEEE 1\u20135.","DOI":"10.1109\/ICCE59016.2024.10444331"},{"key":"e_1_3_1_14_2","first-page":"727","volume-title":"Proceedings of the 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct","author":"Alexdottir Sasha","year":"2022","unstructured":"Sasha Alexdottir and Xiaosong Yang. 2022. Phantom touch phenomenon as a manifestation of the visual-auditory-tactile synaesthesia and its impact on the users in virtual reality. In Proceedings of the 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct. IEEE, 727\u2013732."},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544999.3550158"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.124.3.372"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.3758\/BF03195475"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.psych.50.1.243"},{"issue":"4","key":"e_1_3_1_19_2","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1177\/002224379903600403","article-title":"Visual attention to repeated print advertising: A test of scanpath theory","volume":"36","author":"Pieters Rik","year":"1999","unstructured":"Rik Pieters, Edward Rosbergen, and Michel Wedel. 1999. Visual attention to repeated print advertising: A test of scanpath theory. Journal of Marketing Research 36, 4 (1999), 424\u2013438.","journal-title":"Journal of Marketing Research"},{"key":"e_1_3_1_20_2","doi-asserted-by":"crossref","first-page":"609","DOI":"10.1016\/B978-044451020-4\/50032-3","volume-title":"Proceedings of the Mind\u2019s Eye","author":"Radach Ralph","year":"2003","unstructured":"Ralph Radach, Stefanie Lemmer, Christian Vorstius, Dieter Heller, and Karina Radach. 2003. Eye movements in the processing of print advertisements. In Proceedings of the Mind\u2019s Eye. Elsevier, 609\u2013632."},{"key":"e_1_3_1_21_2","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1016\/B978-044451020-4\/50031-1","volume-title":"Proceedings of the Mind\u2019s Eye","author":"Jacob Robert JK","year":"2003","unstructured":"Robert JK Jacob and Keith S. Karn. 2003. Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. In Proceedings of the Mind\u2019s Eye. Elsevier, 573\u2013605."},{"key":"e_1_3_1_22_2","doi-asserted-by":"crossref","DOI":"10.2501\/JAR-2018-036","article-title":"How advertisers can keep mobile users engaged and reduce video-ad blocking: Best practices for video-ad placement and delivery based on consumer neuroscience measures","author":"Clark Kimberly Rose","year":"2018","unstructured":"Kimberly Rose Clark, Kenneth Raj Leslie, Manuel Garcia-Garcia, and Matthew L. Tullman. 2018. How advertisers can keep mobile users engaged and reduce video-ad blocking: Best practices for video-ad placement and delivery based on consumer neuroscience measures. Journal of Advertising Research 58, 3 (2018), 311\u2013325.","journal-title":"Journal of Advertising Research"},{"key":"e_1_3_1_23_2","doi-asserted-by":"crossref","DOI":"10.2501\/JAR-2023-006","article-title":"Disruptive versus nondisruptive advertising in online streaming video services: How does advertisement placement affect consumer perceptions and ad effectiveness?","author":"Christy Katheryn R.","year":"2023","unstructured":"Katheryn R. Christy, Ranran Z. Mi, Ran Tao, and Linqi Lu. 2023. Disruptive versus nondisruptive advertising in online streaming video services: How does advertisement placement affect consumer perceptions and ad effectiveness? Journal of Advertising Research 63, 2 (2023), 12\u2013138.","journal-title":"Journal of Advertising Research"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397324"},{"key":"e_1_3_1_25_2","unstructured":"Murhaf Hossari Soumyabrata Dev Matthew Nicholson Killian McCabe Atul Nautiyal Clare Conran Jian Tang Wei Xu and Fran\u00e7ois Piti\u00e9. 2018. ADNet: A deep network for detecting adverts. arXiv:1811.04115. Retrieved from https:\/\/arxiv.org\/abs\/1811.04115"},{"key":"e_1_3_1_26_2","doi-asserted-by":"crossref","first-page":"663","DOI":"10.1007\/978-3-030-10997-4_47","volume-title":"Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10\u201314, 2018, Proceedings, Part III 18","author":"Nautiyal Atul","year":"2019","unstructured":"Atul Nautiyal, Killian McCabe, Murhaf Hossari, Soumyabrata Dev, Matthew Nicholson, Clare Conran, Declan McKibben, Jian Tang, Wei Xu, and Fran\u00e7ois Piti\u00e9. 2019. An advert creation system for next-gen publicity. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10\u201314, 2018, Proceedings, Part III 18. Springer, 663\u2013667."},{"key":"e_1_3_1_27_2","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1109\/MMSP.2006.285351","volume-title":"Proceedings of the 2006 IEEE Workshop on Multimedia Signal Processing","author":"Covell Michele","year":"2006","unstructured":"Michele Covell, Shumeet Baluja, and Michael Fink. 2006. Advertisement detection and replacement using acoustic and visual repetition. In Proceedings of the 2006 IEEE Workshop on Multimedia Signal Processing. IEEE, 461\u2013466."},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.123"},{"key":"e_1_3_1_29_2","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1007\/978-981-15-4095-0_2","article-title":"Introduction to reinforcement learning","author":"Ding Zihan","year":"2020","unstructured":"Zihan Ding, Yanhua Huang, Hang Yuan, and Hao Dong. 2020. Introduction to reinforcement learning. Deep Reinforcement Learning: Fundamentals, Research and Applications (2020), 47\u2013123.","journal-title":"Deep Reinforcement Learning: Fundamentals, Research and Applications"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.5555\/3091622.3091655"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553504"},{"key":"e_1_3_1_32_2","first-page":"1040","article-title":"Recommendations with negative feedback via pairwise deep reinforcement learning","author":"Zhao Xiangyu","year":"2018","unstructured":"Xiangyu Zhao, Liang Zhang, Zhuoye Ding, Long Xia, Jiliang Tang, and Dawei Yin. 2018. Recommendations with negative feedback via pairwise deep reinforcement learning. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.1040\u20131048.","journal-title":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining."},{"key":"e_1_3_1_33_2","unstructured":"Xiangyu Zhao Liang Zhang Long Xia Zhuoye Ding Dawei Yin and Jiliang Tang. 2017. Deep reinforcement learning for list-wise recommendations. arXiv:1801.00209. Retrieved from https:\/\/arxiv.org\/abs\/1801.00209"},{"key":"e_1_3_1_34_2","first-page":"1883","article-title":"Whole-chain recommendations","author":"Zhao Xiangyu","year":"2020","unstructured":"Xiangyu Zhao, Long Xia, Lixin Zou, Hui Liu, Dawei Yin, and Jiliang Tang. 2020. Whole-chain recommendations. Proceedings of the 29th ACM International Conference on Information and Knowledge Management.1883\u20131891.","journal-title":"Proceedings of the 29th ACM International Conference on Information and Knowledge Management."},{"key":"e_1_3_1_35_2","doi-asserted-by":"crossref","unstructured":"Xiangyu Zhao Long Xia Lixin Zou Dawei Yin and Jiliang Tang. 2019. Toward simulating environments in reinforcement learning based recommendations. arXiv:1906.11462. Retrieved from https:\/\/arxiv.org\/abs\/1906.11462","DOI":"10.1145\/3320496.3320500"},{"key":"e_1_3_1_36_2","doi-asserted-by":"crossref","first-page":"17","DOI":"10.62051\/ijcsit.v1n1.03","article-title":"Based on intelligent advertising recommendation and abnormal advertising monitoring system in the field of machine learning","author":"Liu Bo","year":"2023","unstructured":"Bo Liu. 2023. Based on intelligent advertising recommendation and abnormal advertising monitoring system in the field of machine learning. International Journal of Computer Science and Information Technology 1, 1 (2023), 17\u201323.","journal-title":"International Journal of Computer Science and Information Technology"},{"key":"e_1_3_1_37_2","doi-asserted-by":"crossref","first-page":"30","DOI":"10.62051\/ijcsit.v1n1.05","article-title":"The implementation of an AI-driven advertising push system based on a NLP algorithm","author":"Xin Qi","year":"2023","unstructured":"Qi Xin, Yuhang He, Yiming Pan, Yong Wang, and Shuqian Du. 2023. The implementation of an AI-driven advertising push system based on a NLP algorithm. International Journal of Computer Science and Information Technology 1, 1 (2023), 30\u201337.","journal-title":"International Journal of Computer Science and Information Technology"},{"key":"e_1_3_1_38_2","doi-asserted-by":"crossref","first-page":"750","DOI":"10.1609\/aaai.v35i1.16156","article-title":"Dear: Deep reinforcement learning for online advertising impression in recommender systems","author":"Zhao Xiangyu","year":"2021","unstructured":"Xiangyu Zhao, Changsheng Gu, Haoshenglun Zhang, Xiwang Yang, Xiaobing Liu, Jiliang Tang, and Hui Liu. 2021. Dear: Deep reinforcement learning for online advertising impression in recommender systems. Proceedings of the AAAI Conference on Artificial Intelligence 35 (2021), 750\u2013758.","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"e_1_3_1_39_2","first-page":"661","article-title":"Real-time bidding by reinforcement learning in display advertising","author":"Cai Han","year":"2017","unstructured":"Han Cai, Kan Ren, Weinan Zhang, Kleanthis Malialis, Jun Wang, Yong Yu, and Defeng Guo. 2017. Real-time bidding by reinforcement learning in display advertising. Proceedings of the 10th ACM International Conference on Web Search and Data Mining.661\u2013670.","journal-title":"Proceedings of the 10th ACM International Conference on Web Search and Data Mining."},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2793599"},{"key":"e_1_3_1_41_2","article-title":"Advertising impression resource allocation strategy with multi-level budget constraint DQN in real-time bidding","volume":"488","author":"Zheng Yu Tian Wanli Xue Tianpei Yang Dou An Yongqi Pi Rong Chen Chengwei Zhang, Kangjie","year":"2022","unstructured":"Yu Tian Wanli Xue Tianpei Yang Dou An Yongqi Pi Rong Chen Chengwei Zhang, Kangjie Zheng. 2022. Advertising impression resource allocation strategy with multi-level budget constraint DQN in real-time bidding. Neurocomputing 488 (2022), 647\u2013656.","journal-title":"Neurocomputing"},{"key":"e_1_3_1_42_2","article-title":"Mask r-cnn","author":"He Kaiming","year":"2017","unstructured":"Kaiming He, Georgia Gkioxari, Piotr Doll\u00e1r, and Ross Girshick. 2017. Mask r-cnn. IEEE International Conference on Computer Vision (2017), 2961\u20132969.","journal-title":"IEEE International Conference on Computer Vision"}],"container-title":["ACM Transactions on Sensor Networks"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3744647","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T14:47:16Z","timestamp":1789742836000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3744647"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,18]]},"references-count":41,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,9,30]]}},"alternative-id":["10.1145\/3744647"],"URL":"https:\/\/doi.org\/10.1145\/3744647","relation":{},"ISSN":["1550-4859","1550-4867"],"issn-type":[{"value":"1550-4859","type":"print"},{"value":"1550-4867","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,18]]},"assertion":[{"value":"2024-06-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-07","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}