{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T09:31:08Z","timestamp":1780651868127,"version":"3.54.1"},"reference-count":63,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2016,2,17]],"date-time":"2016-02-17T00:00:00Z","timestamp":1455667200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Australian Research Council Projects","award":["FT-130101457 and LP-140100569"],"award-info":[{"award-number":["FT-130101457 and LP-140100569"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2016,4]]},"abstract":"<jats:p>Editing faces in videos is a popular yet challenging task in computer vision and graphics that encompasses various applications, including facial attractiveness enhancement, makeup transfer, face replacement, and expression manipulation. Directly applying the existing warping methods to video face editing has the major problem of temporal incoherence in the synthesized videos, which cannot be addressed by simply employing face tracking techniques or manual interventions, as it is difficult to eliminate the subtly temporal incoherence of the facial feature point localizations in a video sequence. In this article, we propose a temporal-spatial-smooth warping (TSSW) method to achieve a high temporal coherence for video face editing. TSSW is based on two observations: (1) the control lattices are critical for generating warping surfaces and achieving the temporal coherence between consecutive video frames, and (2) the temporal coherence and spatial smoothness of the control lattices can be simultaneously and effectively preserved. Based upon these observations, we impose the temporal coherence constraint on the control lattices on two consecutive frames, as well as the spatial smoothness constraint on the control lattice on the current frame. TSSW calculates the control lattice (in either the horizontal or vertical direction) by updating the control lattice (in the corresponding direction) on its preceding frame, i.e., minimizing a novel energy function that unifies a data-driven term, a smoothness term, and feature point constraints. The contributions of this article are twofold: (1) we develop TSSW, which is robust to the subtly temporal incoherence of the facial feature point localizations and is effective to preserve the temporal coherence and spatial smoothness of the control lattices for editing faces in videos, and (2) we present a new unified video face editing framework that is capable for improving the performances of facial attractiveness enhancement, makeup transfer, face replacement, and expression manipulation.<\/jats:p>","DOI":"10.1145\/2819000","type":"journal-article","created":{"date-parts":[[2016,2,22]],"date-time":"2016-02-22T13:07:16Z","timestamp":1456146436000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Video Face Editing Using Temporal-Spatial-Smooth Warping"],"prefix":"10.1145","volume":"7","author":[{"given":"Xiaoyan","family":"Li","sequence":"first","affiliation":[{"name":"University of Technology, Sydney, NSW, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tongliang","family":"Liu","sequence":"additional","affiliation":[{"name":"University of Technology, Sydney, NSW, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiankang","family":"Deng","sequence":"additional","affiliation":[{"name":"University of Technology, Sydney, NSW, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dacheng","family":"Tao","sequence":"additional","affiliation":[{"name":"University of Technology, Sydney, NSW, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2016,2,17]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2614028.2615425"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1006\/cgip.1994.1015"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1399504.1360638"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201912)","author":"Cao Xudong","year":"2012","unstructured":"Xudong Cao , Yichen Wei , Fang Wen , and Jian Sun . 2012 . Face alignment by explicit shape regression . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201912) . 2887--2894. Xudong Cao, Yichen Wei, Fang Wen, and Jian Sun. 2012. Face alignment by explicit shape regression. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201912). 2887--2894."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1186\/1687-5281-2013-13"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.249"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024164"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008122917811"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201910)","author":"Dollar P.","unstructured":"P. Dollar , P. Welinder , and P. Perona . 2010. Cascaded pose regression . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201910) . 1078--1085. P. Dollar, P. Welinder, and P. Perona. 2010. Cascaded pose regression. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201910). 1078--1085."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/645317.649310"},{"key":"e_1_2_1_11_1","first-page":"157","article-title":"Scattered data interpolation: Tests of some methods","volume":"38","author":"Franke Richard","year":"1982","unstructured":"Richard Franke . 1982 . Scattered data interpolation: Tests of some methods . Mathematics of Computation 38 , 157 , 181. Richard Franke. 1982. Scattered data interpolation: Tests of some methods. Mathematics of Computation 38, 157, 181.","journal-title":"Mathematics of Computation"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201909)","author":"Guo Dong","unstructured":"Dong Guo and T. Sim . 2009. Digital face makeup by example . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201909) . 73--79. Dong Guo and T. Sim. 2009. Digital face makeup by example. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201909). 73--79."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-013-0701-3"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2006.171"},{"key":"e_1_2_1_15_1","volume-title":"FDDB: A Benchmark for Face Detection in Unconstrained Settings. Technical Report","author":"Jain V.","year":"2010","unstructured":"V. Jain and E. G. Learned-Miller . 2010 . FDDB: A Benchmark for Face Detection in Unconstrained Settings. Technical Report . University of Massachussetts Amherst , Amherst, MA . V. Jain and E. G. Learned-Miller. 2010. FDDB: A Benchmark for Face Detection in Unconstrained Settings. Technical Report. University of Massachussetts Amherst, Amherst, MA."},{"key":"e_1_2_1_16_1","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV\u201910)","author":"Kemelmacher-Shlizerman Ira","unstructured":"Ira Kemelmacher-Shlizerman , Aditya Sankar , Eli Shechtman , and Steven M. Seitz . 2010. Being John Malkovich . In Proceedings of the European Conference on Computer Vision (ECCV\u201910) . 341--353. Ira Kemelmacher-Shlizerman, Aditya Sankar, Eli Shechtman, and Steven M. Seitz. 2010. Being John Malkovich. In Proceedings of the European Conference on Computer Vision (ECCV\u201910). 341--353."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2000486.2000488"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/2945.620490"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/2945.556502"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360637"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2589481"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508037.2508039"},{"key":"e_1_2_1_23_1","volume-title":"Retrieved","author":"Li Xiaoyan","year":"2014","unstructured":"Xiaoyan Li , Tongliang Liu , Jiankang Deng , and Dacheng Tao . 2014 . Video Face Editing Using Temporal-Spatial-Smooth Warping . Retrieved January 12, 2016, from https:\/\/sites.google.com\/site\/tsswmethod\/. Xiaoyan Li, Tongliang Liu, Jiankang Deng, and Dacheng Tao. 2014. Video Face Editing Using Temporal-Spatial-Smooth Warping. Retrieved January 12, 2016, from https:\/\/sites.google.com\/site\/tsswmethod\/."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2012.26"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.75"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1531326.1531350"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-013-0680-4"},{"key":"e_1_2_1_28_1","first-page":"1","article-title":"Classification with noisy labels by importance reweighting","volume":"99","author":"Liu T.","year":"2015","unstructured":"T. Liu and D. Tao . 2015 . Classification with noisy labels by importance reweighting . IEEE Transactions on Pattern Analysis and Machine Intelligence PP , 99 , 1 . T. Liu and D. Tao. 2015. Classification with noisy labels by importance reweighting. IEEE Transactions on Pattern Analysis and Machine Intelligence PP, 99, 1.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence PP"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1137\/0723042"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2013.2278118"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1137\/0111030"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10593-2_47"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2012.245"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI\u201911)","author":"Nguyen Tuan Anh","year":"2011","unstructured":"Tuan Anh Nguyen , Minh Binh Do , Alfonso Gerevini , Ivan Serina , Biplav Srivastava , and Subbarao Kambhampati . 2011 . Planning with partial preference models . In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI\u201911) . 1772--1777. Tuan Anh Nguyen, Minh Binh Do, Alfonso Gerevini, Ivan Serina, Biplav Srivastava, and Subbarao Kambhampati. 2011. Planning with partial preference models. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI\u201911). 1772--1777."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/38.180119"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.205"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.76"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882269"},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the International Conference on Computer Vision (ICCV\u201907)","author":"Raman S.","unstructured":"S. Raman and S. Chaudhuri . 2007. A matte-less, variational approach to automatic scene compositing . In Proceedings of the International Conference on Computer Vision (ICCV\u201907) . 1--6. S. Raman and S. Chaudhuri. 2007. A matte-less, variational approach to automatic scene compositing. In Proceedings of the International Conference on Computer Vision (ICCV\u201907). 1--6."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2589482"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141920"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2011.01874.x"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-014-0733-3"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.74"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.446"},{"key":"e_1_2_1_46_1","first-page":"1","article-title":"Manifold ranking-based matrix factorization for saliency detection","volume":"99","author":"Tao D.","year":"2015","unstructured":"D. Tao , J. Cheng , M. Song , and X. Lin . 2015 a. Manifold ranking-based matrix factorization for saliency detection . IEEE Transactions on Neural Networks and Learning Systems PP , 99 , 1 . D. Tao, J. Cheng, M. Song, and X. Lin. 2015a. Manifold ranking-based matrix factorization for saliency detection. IEEE Transactions on Neural Networks and Learning Systems PP, 99, 1.","journal-title":"IEEE Transactions on Neural Networks and Learning Systems PP"},{"key":"e_1_2_1_47_1","first-page":"1","article-title":"Principal component 2-D long short-term memory for font recognition on single Chinese characters","volume":"99","author":"Tao D.","year":"2015","unstructured":"D. Tao , X. Lin , L. Jin , and X. Li . 2015 b. Principal component 2-D long short-term memory for font recognition on single Chinese characters . IEEE Transactions on Cybernetics PP , 99 , 1 . D. Tao, X. Lin, L. Jin, and X. Li. 2015b. Principal component 2-D long short-term memory for font recognition on single Chinese characters. IEEE Transactions on Cybernetics PP, 99, 1.","journal-title":"IEEE Transactions on Cybernetics PP"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2008.36"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/PG.2007.21"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2012.105"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.5555\/1756006.1953039"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073209"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778827"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.11"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.75"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2417578"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390334.1390363"},{"key":"e_1_2_1_58_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201912)","author":"Yang Fei","unstructured":"Fei Yang , L. Bourdev , E. Shechtman , Jue Wang , and D. Metaxas . 2012a. Facial expression editing in video using a temporally-smooth factorization . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201912) . 861--868. Fei Yang, L. Bourdev, E. Shechtman, Jue Wang, and D. Metaxas. 2012a. Facial expression editing in video using a temporally-smooth factorization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201912). 861--868."},{"key":"e_1_2_1_59_1","volume-title":"Proceedings of the Graphics Interface Conference (GI\u201912)","author":"Yang Fei","year":"2012","unstructured":"Fei Yang , Eli Shechtman , Jue Wang , Lubomir Bourdev , and Dimitris Metaxas . 2012 b. Face morphing using 3D-aware appearance optimization . In Proceedings of the Graphics Interface Conference (GI\u201912) . 93--99. Fei Yang, Eli Shechtman, Jue Wang, Lubomir Bourdev, and Dimitris Metaxas. 2012b. Face morphing using 3D-aware appearance optimization. In Proceedings of the Graphics Interface Conference (GI\u201912). 93--99."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964955"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-014-0763-x"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10605-2_1"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2013.58"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2819000","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2819000","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T05:07:17Z","timestamp":1750223237000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2819000"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,2,17]]},"references-count":63,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2016,4]]}},"alternative-id":["10.1145\/2819000"],"URL":"https:\/\/doi.org\/10.1145\/2819000","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,2,17]]},"assertion":[{"value":"2014-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-02-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}