{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,18]],"date-time":"2026-01-18T13:57:58Z","timestamp":1768744678699,"version":"3.49.0"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2024,3,27]],"date-time":"2024-03-27T00:00:00Z","timestamp":1711497600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,27]],"date-time":"2024-03-27T00:00:00Z","timestamp":1711497600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Fund of China","doi-asserted-by":"crossref","award":["61473155"],"award-info":[{"award-number":["61473155"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Facial expression manipulation has gained wide attention and has been applied in various fields, such as film production, electronic games, and short videos. However, existing facial expression manipulation methods often overlook the details of local regions in images, resulting in the failure to preserve local structures and textures of images. To solve this problem, this paper proposes a local semantic segmentation mask-based GAN (LSGAN) to generate fine-grained facial expression images. LSGAN is composed of a semantic mask generator, an adversarial autoencoder, a transformative generator, and an AU-intensity discriminator. Our semantic mask generator generates eye, mouth, and cheek masks of face images. Then, our transformative generator integrates target expression labels and corresponding facial region features to generate a vivid target facial expression image. In this fashion, we can capture expressions from target face images explicitly. Furthermore, an AU-intensity discriminator is designed to capture facial expression variations and evaluate quality of generated images. Extensive experiments demonstrate that our method achieves authentic face images with accurate facial expressions and outperforms state-of-the-art methods qualitatively and quantitatively.<\/jats:p>","DOI":"10.1007\/s40747-024-01401-7","type":"journal-article","created":{"date-parts":[[2024,3,27]],"date-time":"2024-03-27T19:02:21Z","timestamp":1711566141000},"page":"4609-4624","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Semantic prior guided fine-grained facial expression manipulation"],"prefix":"10.1007","volume":"10","author":[{"given":"Tao","family":"Xue","sequence":"first","affiliation":[]},{"given":"Jin","family":"Yan","sequence":"additional","affiliation":[]},{"given":"Deshuai","family":"Zheng","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4098-2339","authenticated-orcid":false,"given":"Yong","family":"Liu","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,3,27]]},"reference":[{"key":"1401_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11063-023-11189-1","volume":"55","author":"X Song","year":"2023","unstructured":"Song X, Wu N, Song S, Stojanovic V (2023) Switching-like event-triggered state estimation for reaction-diffusion neural networks against dos attacks. Neural Process Lett 55:1\u201322","journal-title":"Neural Process Lett"},{"key":"1401_CR2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2023.126498","volume":"550","author":"X Song","year":"2023","unstructured":"Song X, Wu N, Song S, Zhang Y, Stojanovic V (2023) Bipartite synchronization for cooperative-competitive neural networks with reaction-diffusion terms via dual event-triggered mechanism. Neurocomputing 550:126498","journal-title":"Neurocomputing"},{"issue":"6","key":"1401_CR3","doi-asserted-by":"publisher","first-page":"7451","DOI":"10.1007\/s40747-023-01135-y","volume":"9","author":"Z Peng","year":"2023","unstructured":"Peng Z, Song X, Song S, Stojanovic V (2023) Hysteresis quantified control for switched reaction-diffusion systems and its application. Complex Intell Syst 9(6):7451\u20137460","journal-title":"Complex Intell Syst"},{"issue":"2","key":"1401_CR4","doi-asserted-by":"publisher","first-page":"380","DOI":"10.1002\/acs.3529","volume":"37","author":"Z Zhang","year":"2023","unstructured":"Zhang Z, Song X, Sun X, Stojanovic V (2023) Hybrid-driven-based fuzzy secure filtering for nonlinear parabolic partial differential equation systems with cyber attacks. Int J Adapt Control Signal Process 37(2):380\u2013398","journal-title":"Int J Adapt Control Signal Process"},{"key":"1401_CR5","doi-asserted-by":"crossref","unstructured":"Choi Y, Choi M, Kim M, Ha J-W, Kim S, Choo J (2018) Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp 8789\u20138797","DOI":"10.1109\/CVPR.2018.00916"},{"key":"1401_CR6","doi-asserted-by":"crossref","unstructured":"Pumarola A, Agudo A, Martinez AM, Sanfeliu A, Moreno-Noguer F (2018) Ganimation: anatomically-aware facial animation from a single image. In Proceedings of the European conference on computer vision (ECCV). pp 818\u2013833","DOI":"10.1007\/978-3-030-01249-6_50"},{"key":"1401_CR7","doi-asserted-by":"crossref","unstructured":"Wegrzyn M, Vogt M, Kireclioglu B, Schneider J, Kissler J (2017) Mapping the emotional face. how individual face parts contribute to successful emotion recognition. PLoS ONE, 12(5):e0177239","DOI":"10.1371\/journal.pone.0177239"},{"key":"1401_CR8","doi-asserted-by":"crossref","unstructured":"Luo C, Song S, Xie W, Shen L, Gunes H (2022) Learning multi-dimensional edge feature-based AU relation graph for facial action unit recognition. In: proceedings of the thirty-first international joint conference on artificial intelligence. International joint conferences on artificial intelligence organization","DOI":"10.24963\/ijcai.2022\/173"},{"key":"1401_CR9","doi-asserted-by":"crossref","unstructured":"Chauhan S, Singh M, Aggarwal AK (2021) Experimental analysis of effect of tuning parameters on the performance of diversity-driven multi-parent evolutionary algorithm. In: 2021 IEEE 2Nd international conference on electrical power and energy systems (ICEPES), pp 1\u20136","DOI":"10.1109\/ICEPES52894.2021.9699655"},{"key":"1401_CR10","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2022.105803","volume":"119","author":"S Chauhan","year":"2023","unstructured":"Chauhan S, Singh M, Aggarwal AK (2023) Designing of optimal digital IIR filter in the multi-objective framework using an evolutionary algorithm. Eng Appl Artif Intell 119:105803","journal-title":"Eng Appl Artif Intell"},{"key":"1401_CR11","doi-asserted-by":"crossref","unstructured":"Yang S, Luo P, Loy CC, Tang X (2016) From facial parts responses to face detection: A deep learning approach. In: IEEE International Conference on Computer Vision","DOI":"10.1109\/ICCV.2015.419"},{"issue":"10","key":"1401_CR12","doi-asserted-by":"publisher","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","volume":"23","author":"K Zhang","year":"2016","unstructured":"Zhang K, Zhang Z, Li Z, Qiao Y (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett 23(10):1499\u20131503","journal-title":"IEEE Signal Process Lett"},{"issue":"1","key":"1401_CR13","doi-asserted-by":"publisher","first-page":"241","DOI":"10.46300\/91011.2022.16.30","volume":"16","author":"AK Aggarwal","year":"2022","unstructured":"Aggarwal AK (2022) Biological tomato leaf disease classification using deep learning framework. Int J Biol Biomed Eng 16(1):241\u2013244","journal-title":"Int J Biol Biomed Eng"},{"key":"1401_CR14","unstructured":"Aggarwal AK, Jaidka P (2022) Segmentation of crop images for crop yield prediction. Int J Biol Biomed 7"},{"key":"1401_CR15","doi-asserted-by":"crossref","unstructured":"Zhao H, Shi J, Qi X, Wang X, Jia J (2016) Pyramid scene parsing network. In: IEEE Computer Society","DOI":"10.1109\/CVPR.2017.660"},{"key":"1401_CR16","doi-asserted-by":"crossref","unstructured":"Lin G, Milan A, Shen C, Reid I (2017) Refinenet: multi-path refinement networks for high-resolution semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1925\u20131934","DOI":"10.1109\/CVPR.2017.549"},{"key":"1401_CR17","volume-title":"Bisenet: bilateral segmentation network for real-time semantic segmentation","author":"C Yu","year":"2018","unstructured":"Yu C, Wang J, Peng C, Gao C, Yu G, Sang N (2018) Bisenet: bilateral segmentation network for real-time semantic segmentation. Springer, Cham"},{"issue":"2021","key":"1401_CR18","first-page":"3051","volume":"129","author":"C Yu","year":"2020","unstructured":"Yu C, Gao C, Wang J, Yu G, Shen C, Sang N (2020) Bisenet v2: bilateral network with guided aggregation for real-time semantic segmentation. Int J Comput Vis 129(2021):3051\u20133068","journal-title":"Int J Comput Vis"},{"issue":"11","key":"1401_CR19","doi-asserted-by":"publisher","first-page":"5464","DOI":"10.1109\/TIP.2019.2916751","volume":"28","author":"Z He","year":"2019","unstructured":"He Z, Zuo W, Kan M, Shan S, Chen X (2019) Attgan: facial attribute editing by only changing what you want. IEEE Trans Image Process 28(11):5464\u20135478","journal-title":"IEEE Trans Image Process"},{"key":"1401_CR20","doi-asserted-by":"crossref","unstructured":"Ding H, Sricharan K, Chellappa R (2018) Exprgan: facial expression editing with controllable expression intensity. In: Proceedings of the AAAI conference on artificial intelligence, vol 32","DOI":"10.1609\/aaai.v32i1.12277"},{"key":"1401_CR21","doi-asserted-by":"publisher","first-page":"2744","DOI":"10.1007\/s11263-020-01361-8","volume":"128","author":"Z Geng","year":"2020","unstructured":"Geng Z, Cao C, Tulyakov S (2020) Towards photo-realistic facial expression manipulation. Int J Comput Vis 128:2744\u20132761","journal-title":"Int J Comput Vis"},{"issue":"3","key":"1401_CR22","doi-asserted-by":"publisher","first-page":"1443","DOI":"10.1109\/TCSVT.2021.3074032","volume":"32","author":"Y Xia","year":"2021","unstructured":"Xia Y, Zheng W, Wang Y, Hui Y, Dong J, Wang F-Y (2021) Local and global perception generative adversarial network for facial expression synthesis. IEEE Trans Circ Syst Video Technol 32(3):1443\u20131452","journal-title":"IEEE Trans Circ Syst Video Technol"},{"key":"1401_CR23","first-page":"1","volume":"525","author":"JF Cohn","year":"2005","unstructured":"Cohn JF, Ekman P (2005) Measuring facial action. New Handbook Methods Nonverbal Behav Res 525:1","journal-title":"New Handbook Methods Nonverbal Behav Res"},{"key":"1401_CR24","doi-asserted-by":"crossref","unstructured":"Baltrusaitis T, Zadeh A, Lim YC, Morency L-P (2018) Openface 2.0: Facial behavior analysis toolkit. In: 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018), pp 59\u201366,","DOI":"10.1109\/FG.2018.00019"},{"key":"1401_CR25","doi-asserted-by":"crossref","unstructured":"Wang J, Zhang J, Lu Z, Shan S (2019) Dft-net: disentanglement of face deformation and texture synthesis for expression editing. In: 2019 IEEE International Conference on Image Processing (ICIP), pp 3881\u20133885","DOI":"10.1109\/ICIP.2019.8803416"},{"key":"1401_CR26","doi-asserted-by":"crossref","unstructured":"Wu R, Zhang G, Lu S, Chen T (2020) Cascade ef-gan: Progressive facial expression editing with local focuses. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp 5021\u20135030","DOI":"10.1109\/CVPR42600.2020.00507"},{"key":"1401_CR27","doi-asserted-by":"crossref","unstructured":"Song L, Lu Z, He R, Sun Z, Tan T (2018) Geometry guided adversarial facial expression synthesis. In: Proceedings of the 26th ACM international conference on Multimedia. pp 627\u201363,","DOI":"10.1145\/3240508.3240612"},{"issue":"3\u20134","key":"1401_CR28","doi-asserted-by":"publisher","DOI":"10.1002\/cav.1819","volume":"29","author":"F Qiao","year":"2018","unstructured":"Qiao F, Yao N, Jiao Z, Li Z, Chen H, Wang H (2018) Emotional facial expression transfer from a single image via generative adversarial nets. Comput Anim Vir Worlds 29(3\u20134):e1819","journal-title":"Comput Anim Vir Worlds"},{"key":"1401_CR29","doi-asserted-by":"crossref","unstructured":"Ling J, Xue H, Song L, Yang S, Xie R, Gu X (2020). Toward fine-grained facial expression manipulation. In: Computer Vision\u2013ECCV 2020: 16th European Conference, Glasgow, UK, August 23\u201328, 2020, Proceedings, Part XXVIII 16, Springer, pp 37\u201353","DOI":"10.1007\/978-3-030-58604-1_3"},{"key":"1401_CR30","doi-asserted-by":"crossref","unstructured":"Wang F, Xiang S, Liu T, Fu Y (2021) Attention based facial expression manipulation. In: 2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp 1\u20136","DOI":"10.1109\/ICMEW53276.2021.9456007"},{"key":"1401_CR31","doi-asserted-by":"publisher","first-page":"4610","DOI":"10.1109\/TIP.2021.3073857","volume":"30","author":"Z Shao","year":"2021","unstructured":"Shao Z, Zhu H, Tang J, Xuequan L, Ma L (2021) Explicit facial expression transfer via fine-grained representations. IEEE Trans Image Process 30:4610\u20134621","journal-title":"IEEE Trans Image Process"},{"issue":"3","key":"1401_CR32","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1109\/MMUL.2021.3061544","volume":"28","author":"J Tang","year":"2021","unstructured":"Tang J, Shao Z, Ma L (2021) Eggan: Learning latent space for fine-grained expression manipulation. IEEE Multimed 28(3):42\u201351","journal-title":"IEEE Multimed"},{"key":"1401_CR33","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"1401_CR34","doi-asserted-by":"crossref","unstructured":"Hou Q, Zhou D, Feng J (2021) Coordinate attention for efficient mobile network design. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. pp 13713\u20131372","DOI":"10.1109\/CVPR46437.2021.01350"},{"key":"1401_CR35","unstructured":"Makhzani A, Shlens J, Jaitly N, Goodfellow I, Frey B (2015) Adversarial autoencoders. arXiv preprint arXiv:1511.05644"},{"key":"1401_CR36","unstructured":"Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556"},{"issue":"4","key":"1401_CR37","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1109\/TIP.2003.819861","volume":"13","author":"Z Wang","year":"2004","unstructured":"Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600\u2013612","journal-title":"IEEE Trans Image Process"},{"key":"1401_CR38","doi-asserted-by":"crossref","unstructured":"Langner O, Dotsch R, Bijlstra G, Wigboldus Daniel HJ, Hawk Skyler T, Knippenberg AD\u00a0Van (2010) Presentation and validation of the radboud faces database. Cogn Emotion 24(8):1377\u20131388","DOI":"10.1080\/02699930903485076"},{"issue":"2","key":"1401_CR39","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1109\/T-AFFC.2013.4","volume":"4","author":"SM Mavadati","year":"2013","unstructured":"Mavadati SM, Mahoor MH, Bartlett K, Trinh P, Cohn JF (2013) Disfa: a spontaneous facial action intensity database. IEEE Trans Affect Comput 4(2):151\u2013160","journal-title":"IEEE Trans Affect Comput"},{"issue":"4","key":"1401_CR40","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1109\/TIP.2003.819861","volume":"13","author":"Z Wang","year":"2004","unstructured":"Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600\u2013612","journal-title":"IEEE Trans Image Process"},{"key":"1401_CR41","unstructured":"Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. Adv Neural Inf Process Syst 30"},{"key":"1401_CR42","doi-asserted-by":"crossref","unstructured":"Zhang R, Isola P, Efros AA, Shechtman E, Wang O (2018) The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 586\u2013595","DOI":"10.1109\/CVPR.2018.00068"},{"key":"1401_CR43","doi-asserted-by":"crossref","unstructured":"Zeng B, Liu X, Gao S, Liu B, Li H, Liu J, Zhang B (2023) Face animation with an attribute-guided diffusion model. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp 628\u2013637","DOI":"10.1109\/CVPRW59228.2023.00070"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-024-01401-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-024-01401-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-024-01401-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,16]],"date-time":"2024-05-16T18:32:30Z","timestamp":1715884350000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-024-01401-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,27]]},"references-count":43,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["1401"],"URL":"https:\/\/doi.org\/10.1007\/s40747-024-01401-7","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"value":"2199-4536","type":"print"},{"value":"2198-6053","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,27]]},"assertion":[{"value":"9 August 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 February 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 March 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}