{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T03:04:12Z","timestamp":1766113452451,"version":"3.48.0"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,12,1]],"date-time":"2025-12-01T00:00:00Z","timestamp":1764547200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T00:00:00Z","timestamp":1766102400000},"content-version":"vor","delay-in-days":18,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001381","name":"National Research Foundation Singapore","doi-asserted-by":"publisher","award":["AISG4-GC-2023-008-1B"],"award-info":[{"award-number":["AISG4-GC-2023-008-1B"]}],"id":[{"id":"10.13039\/501100001381","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Vis. Intell."],"published-print":{"date-parts":[[2025,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Eye direction plays a crucial role in determining the quality of photographs containing human faces. Images where subjects look in inconsistent directions are often perceived as low-quality and discarded. While state-of-the-art deep generative models such as DragGAN, DragDiffusion, and DragonDiffusion (collectively referred to as Drag\u22c6) offer potential solutions for eye direction transformation, their effectiveness for this specific task remains unexplored. In this work, we systematically investigate the capability of Drag\u00a0\u22c6 for eye direction transformation. Our initial experiments reveal that these models in their original form cannot effectively perform this task. To address this limitation, we construct a specialized dataset (i.e., eye multi-direction dataset (EMDD)) and establish a comprehensive benchmark for evaluating methods through fine-tuning on our curated data. Our analysis demonstrates that fine-tuned models achieve satisfactory results when the angular difference between the directions of the source and target eyes is small. However, we observe significant performance degradation when large directional changes are necessary. Through detailed investigation, we uncover the underlying causes of these limitations and provide insights into the models\u2019 failure modes. To overcome these challenges, we propose the\n                    <jats:italic>edge-localized point selector<\/jats:italic>\n                    and\n                    <jats:italic>zero-latent source region replacement<\/jats:italic>\n                    , which can alleviate the identified limitations. Experimental results demonstrate that our approach achieves substantial performance improvements for eye direction transformation, particularly in scenarios involving large angular changes.\n                  <\/jats:p>","DOI":"10.1007\/s44267-025-00103-z","type":"journal-article","created":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T02:34:47Z","timestamp":1766111687000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Benchmarking Drag\u22c6 for eye direction transformation and beyond"],"prefix":"10.1007","volume":"3","author":[{"given":"Yuxiang","family":"Fu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiakun","family":"Ding","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Renzhi","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qian","family":"Fu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ivor W.","family":"Tsang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ming-Ming","family":"Cheng","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0974-9299","authenticated-orcid":false,"given":"Qing","family":"Guo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,12,19]]},"reference":[{"key":"103_CR1","first-page":"18208","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"O. Avrahami","year":"2022","unstructured":"Avrahami, O., Lischinski, D., & Fried, O. (2022). Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 18208\u201318218). Piscataway: IEEE."},{"key":"103_CR2","first-page":"6007","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"B. Kawar","year":"2023","unstructured":"Kawar, B., Zada, S., Lang, O., Tov, O., Chang, H., Dekel, T., Mosseri, I., & Irani, M. (2023). Imagic: text-based real image editing with diffusion models. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 6007\u20136017). Piscataway: IEEE."},{"key":"103_CR3","volume-title":"Proceedings of the 11th international conference on learning representations","author":"A. Hertz","year":"2023","unstructured":"Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., & Cohen-Or, D. (2023). Prompt-to-prompt image editing with cross-attention control. In Proceedings of the 11th international conference on learning representations. Retrieved November 23, 2025, from https:\/\/openreview.net\/forum?id=vHb9IcHFJK."},{"key":"103_CR4","doi-asserted-by":"publisher","DOI":"10.1007\/s44267-024-00058-7","volume":"2","author":"Z. Li","year":"2024","unstructured":"Li, Z., Lv, X., Yu, W., Liu, Q., Lin, J., & Zhang, S. (2024). Face shape transfer via semantic warping. Visual Intelligence, 2, 26.","journal-title":"Visual Intelligence"},{"key":"103_CR5","doi-asserted-by":"publisher","DOI":"10.1007\/s44267-024-00060-z","volume":"2","author":"K. Guo","year":"2024","unstructured":"Guo, K., Zhao, C., & Wang, J. (2024). A fast mask synthesis method for face recognition. Visual Intelligence, 2, 25.","journal-title":"Visual Intelligence"},{"key":"103_CR6","doi-asserted-by":"publisher","DOI":"10.1007\/s44267-024-00057-8","volume":"2","author":"Y. Yan","year":"2024","unstructured":"Yan, Y., Zhou, Z., Wang, Z., Gao, J., & Yang, X. (2024). DialogueNeRF: towards realistic avatar face-to-face conversation video generation. Visual Intelligence, 2, 24.","journal-title":"Visual Intelligence"},{"key":"103_CR7","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1016\/j.neunet.2021.03.019","volume":"141","author":"C. Ryan","year":"2021","unstructured":"Ryan, C., O\u2019Sullivan, B., Elrasad, A., Cahill, A., Lemley, J., Kielty, P., Posch, C., & Perot, E. (2021). Real-time face & eye tracking and blink detection using event cameras. Neural Networks, 141, 87\u201397.","journal-title":"Neural Networks"},{"key":"103_CR8","first-page":"1","volume-title":"Proceedings of the 2021 ACM symposium on eye tracking research and applications","author":"I. Demir","year":"2021","unstructured":"Demir, I., & Ciftci, U. A. (2021). Where do deep fakes look? Synthetic face detection via gaze tracking. In Proceedings of the 2021 ACM symposium on eye tracking research and applications (pp. 1\u201311). New York: ACM."},{"key":"103_CR9","doi-asserted-by":"publisher","first-page":"3010","DOI":"10.1109\/TNNLS.2018.2865525","volume":"30","author":"D. Lian","year":"2018","unstructured":"Lian, D., Hu, L., Luo, W., Xu, Y., Duan, L., Yu, J., & Gao, S. (2018). Multiview multitask gaze estimation with deep convolutional neural networks. IEEE Transactions on Neural Networks and Learning Systems, 30, 3010\u20133023.","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"103_CR10","first-page":"2488","volume-title":"Proceedings of the 33rd AAAI conference on artificial intelligence","author":"D. Lian","year":"2019","unstructured":"Lian, D., Zhang, Z., Luo, W., Hu, L., Wu, M., Li, Z., Yu, J., & Gao, S. (2019). RGBD based gaze estimation via multi-task CNN. In Proceedings of the 33rd AAAI conference on artificial intelligence (pp. 2488\u20132495). Palo Alto: AAAI Press."},{"key":"103_CR11","first-page":"35","volume-title":"Proceedings of the 14th Asian conference on computer vision","author":"D. Lian","year":"2018","unstructured":"Lian, D., Yu, Z., & Gao, S. (2018). Believe it or not, we know what you are looking at! In C. V. Jawahar, H. Li, G. Mori, & K. Schindler (Eds.), Proceedings of the 14th Asian conference on computer vision (pp. 35\u201350). Cham: Springer."},{"key":"103_CR12","doi-asserted-by":"publisher","DOI":"10.3390\/s19245540","volume":"19","author":"M. Q. Khan","year":"2019","unstructured":"Khan, M. Q., & Lee, S. (2019). Gaze and eye tracking: techniques and applications in ADAS. Sensors, 19, 5540.","journal-title":"Sensors"},{"key":"103_CR13","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2024.108117","volume":"133","author":"P. K. Sharma","year":"2024","unstructured":"Sharma, P. K., & Chakraborty, P. (2024). A review of driver gaze estimation and application in gaze behavior understanding. Engineering Applications of Artificial Intelligence, 133, 108117.","journal-title":"Engineering Applications of Artificial Intelligence"},{"key":"103_CR14","doi-asserted-by":"publisher","DOI":"10.1016\/j.jesp.2019.103842","volume":"85","author":"R.-M. Rahal","year":"2019","unstructured":"Rahal, R.-M., & Fiedler, S. (2019). Understanding cognitive and affective mechanisms in social psychology through eye-tracking. Journal of Experimental Social Psychology, 85, 103842.","journal-title":"Journal of Experimental Social Psychology"},{"key":"103_CR15","first-page":"936","volume-title":"Proceedings of the IFIP conference on human-computer interaction","author":"T. Strandvall","year":"2009","unstructured":"Strandvall, T. (2009). Eye tracking in human-computer interaction and usability research. In Proceedings of the IFIP conference on human-computer interaction. (pp. 936\u2013937). Cham: Springer."},{"key":"103_CR16","first-page":"1","volume-title":"Proceedings of the ACM SIGGRAPH 2023 conference","author":"X. Pan","year":"2023","unstructured":"Pan, X., Tewari, A., Leimk\u00fchler, T., Liu, L., Meka, A., & Theobalt, C. (2023). Drag your GAN: interactive point-based manipulation on the generative image manifold. In Proceedings of the ACM SIGGRAPH 2023 conference (pp. 1\u201311). New York: ACM."},{"key":"103_CR17","first-page":"8839","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"Y. Shi","year":"2024","unstructured":"Shi, Y., Xue, C., Liew, J. H., Pan, J., Yan, H., Zhang, W., Tan, V. Y., & Bai, S. (2024). DragDiffusion: harnessing diffusion models for interactive point-based image editing. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 8839\u20138849). Piscataway: IEEE."},{"key":"103_CR18","volume-title":"Proceedings of the 12th international conference on learning representations","author":"C. Mou","year":"2023","unstructured":"Mou, C., Wang, X., Song, J., Shan, Y., & Zhang, J. (2023). DragonDiffusion: enabling drag-style manipulation on diffusion models. In Proceedings of the 12th international conference on learning representations (pp. 1-12). Retrieved November 23, 2025, from https:\/\/openreview.net\/forum?id=OEL4FJMg1b."},{"key":"103_CR19","doi-asserted-by":"publisher","first-page":"271","DOI":"10.1145\/2501988.2501994","volume-title":"Proceedings of the 26th annual ACM symposium on user interface software and technology","author":"B. A. Smith","year":"2013","unstructured":"Smith, B. A., Yin, Q., Feiner, S. K., & Nayar, S. K. (2013). Gaze locking: passive eye contact detection for human-object interaction. In Proceedings of the 26th annual ACM symposium on user interface software and technology (pp. 271\u2013280). New York: ACM."},{"key":"103_CR20","first-page":"8110","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"T. Karras","year":"2020","unstructured":"Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020). Analyzing and improving the image quality of styleGAN. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 8110\u20138119). Piscataway: IEEE."},{"key":"103_CR21","first-page":"6840","volume-title":"Proceedings of the 34th international conference on neural information processing systems","author":"J. Ho","year":"2020","unstructured":"Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, & H. Lin (Eds.), Proceedings of the 34th international conference on neural information processing systems. (pp. 6840\u20136851). Red Hook: Curran Associates."},{"key":"103_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2366145.2366193","volume":"31","author":"C. Kuster","year":"2012","unstructured":"Kuster, C., Popa, T., Bazin, J.-C., Gotsman, C., & Gross, M. (2012). Gaze correction for home video conferencing. ACM Transactions on Graphics, 31, 1\u20136.","journal-title":"ACM Transactions on Graphics"},{"key":"103_CR23","first-page":"8789","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"Y. Choi","year":"2018","unstructured":"Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018). StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 8789\u20138797). Piscataway: IEEE."},{"key":"103_CR24","first-page":"852","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"H. Kim","year":"2021","unstructured":"Kim, H., Choi, Y., Kim, J., Yoo, S., & Uh, Y. (2021). Exploiting spatial dimensions of latent in GAN for real-time image editing. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 852\u2013861). Piscataway: IEEE."},{"key":"103_CR25","first-page":"11254","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"Y. Shi","year":"2022","unstructured":"Shi, Y., Yang, X., Wan, Y., & Shen, X. (2022). SemanticStyleGAN: learning compositional generative priors for controllable image synthesis and editing. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 11254\u201311264). Piscataway: IEEE."},{"key":"103_CR26","first-page":"10765","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"H. Liu","year":"2021","unstructured":"Liu, H., Wan, Z., Huang, W., Song, Y., Han, X., Liao, J., Jiang, B., & Liu, W. (2021). DeFLOCNet: deep image editing via flexible low-level controls. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 10765\u201310774). Piscataway: IEEE."},{"key":"103_CR27","first-page":"1003","volume-title":"Proceedings of the IEEE international conference on computer vision","author":"K. Wang","year":"2017","unstructured":"Wang, K., & Ji, Q. (2017). Real time eye gaze tracking with 3D deformable eye-face model. In Proceedings of the IEEE international conference on computer vision (pp. 1003\u20131011). Piscataway: IEEE."},{"key":"103_CR28","first-page":"191","volume-title":"Proceedings 9th IEEE international conference on computer vision","author":"A. Criminisi","year":"2003","unstructured":"Criminisi, A., Shotton, J., Blake, A., & Torr, P. (2003). Gaze manipulation for one-to-one teleconferencing. In Proceedings 9th IEEE international conference on computer vision (pp. 191\u2013198). Piscataway: IEEE."},{"key":"103_CR29","first-page":"479","volume-title":"Proceedings of the 7th European conference on computer vision","author":"R. Yang","year":"2002","unstructured":"Yang, R., & Zhang, Z. (2002). Eye gaze correction with stereovision for video-teleconferencing. In A. Heyden, G. Sparr, M. Nielsen, & P. Johansen (Eds.), Proceedings of the 7th European conference on computer vision (pp. 479\u2013494). Cham: Springer."},{"key":"103_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/3DRes.03(2011)5","volume":"2","author":"J. Zhu","year":"2011","unstructured":"Zhu, J., Yang, R., & Xiang, X. (2011). Eye contact in video conference via fusion of time-of-flight depth sensor and stereo. 3D Research, 2, 1\u201310.","journal-title":"3D Research"},{"key":"103_CR31","first-page":"311","volume-title":"Proceedings of the 14th European conference on computer vision","author":"Y. Ganin","year":"2016","unstructured":"Ganin, Y., Kononenko, D., Sungatullina, D., & Lempitsky, V. (2016). Deepwarp: photorealistic image resynthesis for gaze manipulation. In L. Bastian, M. Ji\u0159\u00ed, S. Nicu, & W. Max (Eds.), Proceedings of the 14th European conference on computer vision (pp. 311\u2013326). Cham: Springer."},{"key":"103_CR32","first-page":"4432","volume-title":"Proceedings of the IEEE\/CVF international conference on computer vision","author":"R. Abdal","year":"2019","unstructured":"Abdal, R., Qin, Y., & Wonka, P. (2019). Image2StyleGAN: how to embed images into the stylegan latent space? In Proceedings of the IEEE\/CVF international conference on computer vision (pp. 4432\u20134441). Piscataway: IEEE."},{"key":"103_CR33","first-page":"8296","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"R. Abdal","year":"2020","unstructured":"Abdal, R., Qin, Y., & Wonka, P. (2020). Image2StyleGAN++: how to edit the embedded images? In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 8296\u20138305). Piscataway: IEEE."},{"key":"103_CR34","first-page":"18511","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"Y. Alaluf","year":"2022","unstructured":"Alaluf, Y., Tov, O., Mokady, R., Gal, R., & Bermano, A. (2022). HyperStyle: StyleGAN inversion with hypernetworks for real image editing. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 18511\u201318521). Piscataway: IEEE."},{"key":"103_CR35","volume-title":"Proceedings of the 12th international conference on learning representations","author":"C. Meng","year":"2021","unstructured":"Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.-Y., & Ermon, S. (2021). SDEdit: guided image synthesis and editing with stochastic differential equations. In Proceedings of the 12th international conference on learning representations. (pp. 1-33). Retrieved November 23, 2025, from https:\/\/openreview.net\/forum?id=aBsCjcPu_tE."},{"key":"103_CR36","first-page":"18392","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"T. Brooks","year":"2023","unstructured":"Brooks, T., Holynski, A., & Efros, A. A. (2023). InstructPix2Pix: learning to follow image editing instructions. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 18392\u201318402). Piscataway: IEEE."},{"issue":"6","key":"103_CR37","doi-asserted-by":"publisher","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","volume":"8","author":"J. Canny","year":"1986","unstructured":"Canny, J. (1986). A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(6), 679\u2013698.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"}],"container-title":["Visual Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44267-025-00103-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s44267-025-00103-z","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44267-025-00103-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T02:34:52Z","timestamp":1766111692000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s44267-025-00103-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12]]},"references-count":37,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,12]]}},"alternative-id":["103"],"URL":"https:\/\/doi.org\/10.1007\/s44267-025-00103-z","relation":{},"ISSN":["2097-3330","2731-9008"],"issn-type":[{"value":"2097-3330","type":"print"},{"value":"2731-9008","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12]]},"assertion":[{"value":"1 September 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 December 2025","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 December 2025","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 December 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Ming-Ming Cheng and Qing Guo are Associate Editors at Visual Intelligence and were not involved in the editorial review of this article or the decision to publish it. The authors declare that they have no other competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"29"}}