{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T21:44:27Z","timestamp":1773524667379,"version":"3.50.1"},"reference-count":34,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2019,5,31]],"date-time":"2019-05-31T00:00:00Z","timestamp":1559260800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2019,5,31]]},"abstract":"<jats:p>Although live video communication is widely used, it is generally less engaging than face-to-face communication because of limitations on social, emotional, and haptic feedback. Missing eye contact is one such problem caused by the physical deviation between the screen and camera on a device. Manipulating video frames to correct eye gaze is a solution to this problem. In this article, we introduce a system to rotate the eyeball of a local participant before the video frame is sent to the remote side. It adopts a warping-based convolutional neural network to relocate pixels in eye regions. To improve visual quality, we minimize the L2 distance between the ground truths and warped eyes. We also present several newly designed loss functions to help network training. These new loss functions are designed to preserve the shape of eye structures and minimize color changes around the periphery of eye regions. To evaluate the presented network and loss functions, we objectively and subjectively compared results generated by our system and the state-of-the-art, DeepWarp, in relation to two datasets. The experimental results demonstrated the effectiveness of our system. In addition, we showed that our system can perform eye-gaze correction in real time on a consumer-level laptop. Because of the quality and efficiency of the system, gaze correction by postprocessing through this system is a feasible solution to the problem of missing eye contact in video communication.<\/jats:p>","DOI":"10.1145\/3311784","type":"journal-article","created":{"date-parts":[[2019,6,6]],"date-time":"2019-06-06T12:28:42Z","timestamp":1559824122000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Look at Me! Correcting Eye Gaze in Live Video Communication"],"prefix":"10.1145","volume":"15","author":[{"given":"Chih-Fan","family":"Hsu","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, National Taiwan University"}]},{"given":"Yu-Shuen","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, National Chiao Tung University"}]},{"given":"Chin-Laung","family":"Lei","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Taiwan University"}]},{"given":"Kuan-Ta","family":"Chen","sequence":"additional","affiliation":[{"name":"Institute of Information Science, Academia Sinica"}]}],"member":"320","published-online":{"date-parts":[[2019,6,5]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Trends 8 Analysis--Forecasts To","author":"Banerjee T.","year":"2025","unstructured":"T. Banerjee . Webinar 8 Webcast Market Size , Trends 8 Analysis--Forecasts To 2025 . Retrieved from https:\/\/medium.com\/@banerjee.treesha\/webinar-webcast-market-size-trends-analysis-forecasts-to-2025-1877a838ce39. T. Banerjee. Webinar 8 Webcast Market Size, Trends 8 Analysis--Forecasts To 2025. Retrieved from https:\/\/medium.com\/@banerjee.treesha\/webinar-webcast-market-size-trends-analysis-forecasts-to-2025-1877a838ce39."},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","unstructured":"P. S. N. Lee L. Leung V. Lo C. Xiong and T. Wu. 2011. Internet communication versus face-to-face interaction in quality of life. Soc. Indicat. Res. 100 3 (01 Feb. 2011) 375--389.  P. S. N. Lee L. Leung V. Lo C. Xiong and T. Wu. 2011. Internet communication versus face-to-face interaction in quality of life. Soc. Indicat. Res. 100 3 (01 Feb. 2011) 375--389.","DOI":"10.1007\/s11205-010-9618-3"},{"key":"e_1_2_1_3_1","unstructured":"The Late Late Show with James Corden. 2017. Harry Styles video chats with james corden. Retrieved from https:\/\/www.youtube.com\/watch?v&equals;H7ZjRna4ZK4.  The Late Late Show with James Corden. 2017. Harry Styles video chats with james corden. Retrieved from https:\/\/www.youtube.com\/watch?v&equals;H7ZjRna4ZK4."},{"key":"e_1_2_1_4_1","doi-asserted-by":"crossref","unstructured":"Y. Ganin D. Kononenko D. Sungatullina and V. Lempitsky. 2016. DeepWarp: Photorealistic Image Resynthesis for Gaze Manipulation. Springer International Publishing 311--326.  Y. Ganin D. Kononenko D. Sungatullina and V. Lempitsky. 2016. DeepWarp: Photorealistic Image Resynthesis for Gaze Manipulation. Springer International Publishing 311--326.","DOI":"10.1007\/978-3-319-46475-6_20"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201917)","author":"Huang G.","unstructured":"G. Huang , Z. Liu , and K. Q. Weinberger . 2017. Densely connected convolutional networks . In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201917) . 2261--2269. arxiv:1608.06993 http:\/\/arxiv.org\/abs\/1608.06993 G. Huang, Z. Liu, and K. Q. Weinberger. 2017. Densely connected convolutional networks. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201917). 2261--2269. arxiv:1608.06993 http:\/\/arxiv.org\/abs\/1608.06993"},{"key":"e_1_2_1_6_1","unstructured":"R. Yang and Z. Zhang. 2001. Eye Gaze Correction with Stereovision for Video-Teleconferencing. Technical Report. Microsoft. Retrieved from https:\/\/www.microsoft.com\/en-us\/research\/publication\/eye-gaze-correction-with-stereovision-for-video-teleconferencing\/.  R. Yang and Z. Zhang. 2001. Eye Gaze Correction with Stereovision for Video-Teleconferencing. Technical Report. Microsoft. Retrieved from https:\/\/www.microsoft.com\/en-us\/research\/publication\/eye-gaze-correction-with-stereovision-for-video-teleconferencing\/."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/946247.946637"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 817--824","author":"Wolf L.","unstructured":"L. Wolf , Z. Freund , and S. Avidan . 2010. An eye for an eye: A single camera gaze-replacement method . In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 817--824 . L. Wolf, Z. Freund, and S. Avidan. 2010. An eye for an eye: A single camera gaze-replacement method. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 817--824."},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the 33rd International Conference on Information Technology Interfaces (ITI\u201911)","author":"Solina F.","unstructured":"F. Solina and R. Ravnik . 2011. Fixing missing eye-contact in video conferencing systems . In Proceedings of the 33rd International Conference on Information Technology Interfaces (ITI\u201911) . 233--236. F. Solina and R. Ravnik. 2011. Fixing missing eye-contact in video conferencing systems. In Proceedings of the 33rd International Conference on Information Technology Interfaces (ITI\u201911). 233--236."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/93.895152"},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the 2014 IEEE International Conference on Multimedia and Expo (ICME\u201914)","author":"Giger D.","unstructured":"D. Giger , J. C. Bazin , C. Kuster , T. Popa , and M. Gross . 2014. Gaze correction with a single webcam . In Proceedings of the 2014 IEEE International Conference on Multimedia and Expo (ICME\u201914) . 1--6. D. Giger, J. C. Bazin, C. Kuster, T. Popa, and M. Gross. 2014. Gaze correction with a single webcam. In Proceedings of the 2014 IEEE International Conference on Multimedia and Expo (ICME\u201914). 1--6."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.displa.2016.12.002"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.displa.2012.10.009"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11760-016-0918-1"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1037\/1076-898X.3.2.105"},{"key":"e_1_2_1_16_1","volume-title":"Proceedings of the UBICOMP 2003 Video Program.","author":"Tapia E. M.","unstructured":"E. M. Tapia , S. S. Intille , J. R. Rebula , and S. Stoddard . 2003. Concept and partial prototype video: Ubiquitous video communication with the perception of eye contact . In Proceedings of the UBICOMP 2003 Video Program. E. M. Tapia, S. S. Intille, J. R. Rebula, and S. Stoddard. 2003. Concept and partial prototype video: Ubiquitous video communication with the perception of eye contact. In Proceedings of the UBICOMP 2003 Video Program."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1531326.1531370"},{"key":"e_1_2_1_18_1","volume-title":"United States Patent US20160358543A1","author":"Rappoport B. M.","year":"2016","unstructured":"B. M. Rappoport , C. J. Stringer , F. R. Rothkopf , J. C. Franklin , J. P. Ternus , J. C. Hoenig , R. P. Howarth , S. A. MYERS, and S. B. Lynch . 2016. Devices and methods for providing access to internal component . United States Patent US20160358543A1 , 2016 . B. M. Rappoport, C. J. Stringer, F. R. Rothkopf, J. C. Franklin, J. P. Ternus, J. C. Hoenig, R. P. Howarth, S. A. MYERS, and S. B. Lynch. 2016. Devices and methods for providing access to internal component. United States Patent US20160358543A1, 2016."},{"key":"e_1_2_1_19_1","volume-title":"United States Patent US20120069042A1","author":"T.","year":"2012","unstructured":"T. OGITA, S. Takanashi , and S. Takatsuka 2012 . Sensor-equipped display apparatus and electronic apparatus . United States Patent US20120069042A1 , 2012. T. OGITA, S. Takanashi, and S. Takatsuka 2012. Sensor-equipped display apparatus and electronic apparatus. United States Patent US20120069042A1, 2012."},{"key":"e_1_2_1_20_1","doi-asserted-by":"crossref","unstructured":"M. Dumont S. Rogmans S. Maesen and P. Bekaert. 2009. Optimized two-party video chat with restored eye contact using graphics hardware. In e-Business and Telecommunications Joaquim Filipe and Mohammad S. Obaidat (Eds.). Springer Berlin 358--372.  M. Dumont S. Rogmans S. Maesen and P. Bekaert. 2009. Optimized two-party video chat with restored eye contact using graphics hardware. In e-Business and Telecommunications Joaquim Filipe and Mohammad S. Obaidat (Eds.). Springer Berlin 358--372.","DOI":"10.1007\/978-3-642-05197-5_26"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366193"},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the 12th International Conference on Image Analysis and Processing. 76--81","author":"Weiner D.","unstructured":"D. Weiner and N. Kiryati . 2003. Virtual gaze redirection in face images . In Proceedings of the 12th International Conference on Image Analysis and Processing. 76--81 . D. Weiner and N. Kiryati. 2003. Virtual gaze redirection in face images. In Proceedings of the 12th International Conference on Image Analysis and Processing. 76--81."},{"key":"e_1_2_1_23_1","doi-asserted-by":"crossref","unstructured":"Y. Qin K. C. Lien M. Turk and T. H\u00f6llerer. 2015. Eye Gaze Correction with a Single Webcam Based on Eye-Replacement. Springer International Publishing Cham 599--609.  Y. Qin K. C. Lien M. Turk and T. H\u00f6llerer. 2015. Eye Gaze Correction with a Single Webcam Based on Eye-Replacement. Springer International Publishing Cham 599--609.","DOI":"10.1007\/978-3-319-27857-5_54"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2926713"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13355"},{"key":"e_1_2_1_26_1","volume-title":"Computer Vision: A Modern Approach","author":"Forsyth D. A.","year":"2002","unstructured":"D. A. Forsyth and J. Ponce . 2002 . Computer Vision: A Modern Approach . Prentice Hall Professional . D. A. Forsyth and J. Ponce. 2002. Computer Vision: A Modern Approach. Prentice Hall Professional."},{"key":"e_1_2_1_27_1","volume-title":"Variation and extrema of human interpupillary distance","author":"Dodgson N. A.","unstructured":"N. A. Dodgson . 2004. Variation and extrema of human interpupillary distance . In Stereoscopic Displays and Virtual Reality Systems XI, Andrew J. Woods, John O. Merritt, Stephen A. Benton, and Mark T. Bolas (Eds.), Vol. 5291 . SPIE , 19--22. N. A. Dodgson. 2004. Variation and extrema of human interpupillary distance. In Stereoscopic Displays and Virtual Reality Systems XI, Andrew J. Woods, John O. Merritt, Stephen A. Benton, and Mark T. Bolas (Eds.), Vol. 5291. SPIE, 19--22."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/1577069.1755843"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.241"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the ICML Deep Learning Workshop","author":"Xu B.","year":"2015","unstructured":"B. Xu , N. Wang , T. Chen , and M. Li . 2015. Empirical evaluation of rectified activations in convolutional network . In Proceedings of the ICML Deep Learning Workshop ( 2015 ). 06--11. arxiv:1505.00853 http:\/\/arxiv.org\/abs\/1505.00853 B. Xu, N. Wang, T. Chen, and M. Li. 2015. Empirical evaluation of rectified activations in convolutional network. In Proceedings of the ICML Deep Learning Workshop (2015). 06--11. arxiv:1505.00853 http:\/\/arxiv.org\/abs\/1505.00853"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the 32nd International Conference on International Conference on Machine Learning","volume":"37","author":"Ioffe S.","unstructured":"S. Ioffe and C. Szegedy . 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift . In Proceedings of the 32nd International Conference on International Conference on Machine Learning , Vol. 37 . 448--456. http:\/\/dl.acm.org\/citation.cfm?id&equals;3045118.3045167 S. Ioffe and C. Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning, Vol. 37. 448--456. http:\/\/dl.acm.org\/citation.cfm?id&equals;3045118.3045167"},{"key":"e_1_2_1_32_1","unstructured":"M. Abadi A. Agarwal P. Barham E. Brevdo Z. Chen C. Citro G.S. Corrado A. Davis J. Dean M. Devin S. Ghemawat I. Goodfellow A. Harp G. Irving M. Isard Y. Jia R. Jozefowicz L. Kaiser M. Kudlur J. Levenberg D. Man\u00e9 R. Monga S. Moore D. Murray C. Olah M. Schuster J. Shlens B. Steiner I. Sutskever K. Talwar P. Tucker V. Vanhoucke V. Vasudevan F. Vi\u00e9gas O. Vinyals P. Warden M. Wattenberg M. Wicke Y. Yu and X. Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Retrieved from https:\/\/www.tensorflow.org\/.  M. Abadi A. Agarwal P. Barham E. Brevdo Z. Chen C. Citro G.S. Corrado A. Davis J. Dean M. Devin S. Ghemawat I. Goodfellow A. Harp G. Irving M. Isard Y. Jia R. Jozefowicz L. Kaiser M. Kudlur J. Levenberg D. Man\u00e9 R. Monga S. Moore D. Murray C. Olah M. Schuster J. Shlens B. Steiner I. Sutskever K. Talwar P. Tucker V. Vanhoucke V. Vasudevan F. Vi\u00e9gas O. Vinyals P. Warden M. Wattenberg M. Wicke Y. Yu and X. Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Retrieved from https:\/\/www.tensorflow.org\/."},{"key":"e_1_2_1_33_1","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Kingma D. P.","year":"2015","unstructured":"D. P. Kingma and J. Ba . 2015. Adam: A method for stochastic optimization . In Proceedings of the International Conference on Learning Representations ( 2015 ). arxiv:1412.6980 http:\/\/arxiv.org\/abs\/1412.6980 D. P. Kingma and J. Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (2015). arxiv:1412.6980 http:\/\/arxiv.org\/abs\/1412.6980"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2501988.2501994"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3311784","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3311784","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:25:31Z","timestamp":1750206331000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3311784"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,31]]},"references-count":34,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2019,5,31]]}},"alternative-id":["10.1145\/3311784"],"URL":"https:\/\/doi.org\/10.1145\/3311784","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,5,31]]},"assertion":[{"value":"2018-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-02-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-06-05","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}